== sanity test 123ad: Verify batching statahead works correctly ========================================================== 23:39:42 (1789443582) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8a0e369b2800.batch_stats=0 llite.lustre-ffff8a0e369b2800.statahead_max=32 llite.lustre-ffff8a0e369b2800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8a0e369b2800.batch_stats=0 total: 100 open/close in 2.78 seconds: 35.91 ops/second llite.lustre-ffff8a0e369b2800.statahead_max=0 101 real 0m2.718s user 0m0.010s sys 0m0.349s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8a0e369b2800.statahead_max=32 llite.lustre-ffff8a0e369b2800.statahead_max=32 101 real 0m0.930s user 0m0.004s sys 0m0.157s 'ls -l' 100 files with statahead: 1 sec statahead total: 118 statahead wrong: 2 agl total: 112 list_total: 35 fname_total: 60 hit_total: 3932 miss_total: 1664 swrong=2 ewrong=2 - open/close 637 (time 1789443610.06 total 10.01 last 63.66) total: 900 open/close in 14.07 seconds: 63.95 ops/second llite.lustre-ffff8a0e369b2800.statahead_max=0 1001 real 0m22.766s user 0m0.039s sys 0m2.883s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff8a0e369b2800.statahead_max=32 llite.lustre-ffff8a0e369b2800.statahead_max=32 1001 real 0m10.979s user 0m0.029s sys 0m1.345s 'ls -l' 1000 files with statahead: 11 sec statahead total: 119 statahead wrong: 2 agl total: 113 list_total: 36 fname_total: 60 hit_total: 4930 miss_total: 1666 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff8a0e369b2800.statahead_stats= statahead total: 119 statahead wrong: 2 agl total: 113 list_total: 36 fname_total: 60 hit_total: 4930 miss_total: 1666 mdc.lustre-MDT0000-mdc-ffff8a0e369b2800.batch_stats= snapshot_time: 1789443684.844500032 secs.nsecs start_time: 1789443587.312681781 secs.nsecs elapsed_time: 97.531818251 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 605 (time 1789443697.76 total 10.01 last 60.45) - open/close 1244 (time 1789443707.76 total 20.01 last 63.89) - open/close 1891 (time 1789443717.76 total 30.01 last 64.69) - open/close 2510 (time 1789443727.77 total 40.02 last 61.85) - open/close 3208 (time 1789443737.78 total 50.03 last 69.74) - open/close 3886 (time 1789443747.78 total 60.03 last 67.80) - open/close 4566 (time 1789443757.78 total 70.04 last 67.94) - open/close 5208 (time 1789443767.79 total 80.04 last 64.18) - open/close 5856 (time 1789443777.81 total 90.06 last 64.64) - open/close 6458 (time 1789443787.82 total 100.08 last 60.13) - open/close 7104 (time 1789443797.83 total 110.08 last 64.54) - open/close 7763 (time 1789443807.84 total 120.09 last 65.88) - open/close 8354 (time 1789443817.85 total 130.10 last 59.05) - open/close 8974 (time 1789443827.85 total 140.10 last 61.99) - open/close 9619 (time 1789443837.85 total 150.10 last 64.50) total: 10000 open/close in 155.90 seconds: 64.14 ops/second llite.lustre-ffff8a0e369b2800.statahead_batch_max=0 llite.lustre-ffff8a0e369b2800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8a0e369b2800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m58.744s user 0m0.299s sys 0m15.221s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8a0e369b2800.statahead_batch_max=32 llite.lustre-ffff8a0e369b2800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8a0e369b2800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8a0e369b2800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m42.764s user 0m0.300s sys 0m13.381s batched statahead RPC= hit count=9986 mdc.lustre-MDT0000-mdc-ffff8a0e369b2800.batch_stats= snapshot_time: 1789444023.938532188 secs.nsecs start_time: 1789443911.376567499 secs.nsecs elapsed_time: 112.561964689 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8a0e369b2800.batch_stats=0 llite.lustre-ffff8a0e369b2800.statahead_max=2048 llite.lustre-ffff8a0e369b2800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8a0e369b2800.batch_stats=0 total: 100 open/close in 2.96 seconds: 33.83 ops/second llite.lustre-ffff8a0e369b2800.statahead_max=0 101 real 0m2.380s user 0m0.006s sys 0m0.309s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8a0e369b2800.statahead_max=2048 llite.lustre-ffff8a0e369b2800.statahead_max=2048 101 real 0m1.019s user 0m0.007s sys 0m0.147s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10085 miss_total: 15 swrong=0 ewrong=0 - open/close 676 (time 1789444365.30 total 10.01 last 67.52) total: 900 open/close in 13.36 seconds: 67.39 ops/second llite.lustre-ffff8a0e369b2800.statahead_max=0 1001 real 0m21.081s user 0m0.033s sys 0m2.797s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff8a0e369b2800.statahead_max=2048 llite.lustre-ffff8a0e369b2800.statahead_max=2048 1001 real 0m6.996s user 0m0.031s sys 0m1.480s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11084 miss_total: 16 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff8a0e369b2800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11084 miss_total: 16 mdc.lustre-MDT0000-mdc-ffff8a0e369b2800.batch_stats= snapshot_time: 1789444432.983585205 secs.nsecs start_time: 1789444026.547432355 secs.nsecs elapsed_time: 406.436152850 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 22 22 16: 2 22 44 32: 0 0 44 64: 0 0 44 128: 1 11 55 256: 4 44 100 - open/close 715 (time 1789444445.98 total 10.02 last 71.36) - open/close 1359 (time 1789444455.98 total 20.02 last 64.39) - open/close 2000 (time 1789444465.99 total 30.03 last 64.07) - open/close 2625 (time 1789444476.00 total 40.04 last 62.39) - open/close 3302 (time 1789444486.01 total 50.05 last 67.64) - open/close 3981 (time 1789444496.02 total 60.06 last 67.81) - open/close 4655 (time 1789444506.04 total 70.08 last 67.29) - open/close 5302 (time 1789444516.05 total 80.09 last 64.65) - open/close 5954 (time 1789444526.05 total 90.09 last 65.18) - open/close 6630 (time 1789444536.06 total 100.10 last 67.54) - open/close 7308 (time 1789444546.07 total 110.11 last 67.71) - open/close 7988 (time 1789444556.09 total 120.13 last 67.92) - open/close 8576 (time 1789444566.10 total 130.14 last 58.73) - open/close 9128 (time 1789444576.10 total 140.14 last 55.17) - open/close 9739 (time 1789444586.11 total 150.15 last 61.08) total: 10000 open/close in 154.09 seconds: 64.90 ops/second llite.lustre-ffff8a0e369b2800.statahead_batch_max=0 llite.lustre-ffff8a0e369b2800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8a0e369b2800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m54.281s user 0m0.338s sys 0m16.122s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8a0e369b2800.statahead_batch_max=256 llite.lustre-ffff8a0e369b2800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8a0e369b2800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8a0e369b2800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m55.134s user 0m0.336s sys 0m14.705s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8a0e369b2800.batch_stats= snapshot_time: 1789444718.294442728 secs.nsecs start_time: 1789444652.363966644 secs.nsecs elapsed_time: 65.930476084 secs.nsecs subreqs per batch batches %% cum %% 1: 66 36 36 2: 25 13 50 4: 19 10 60 8: 8 4 64 16: 6 3 68 32: 17 9 77 64: 4 2 79 128: 2 1 80 256: 35 19 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8a0e369b2800.statahead_batch_max=64 llite.lustre-ffff8a0e369b2800.statahead_max=128