== sanity test 123ad: Verify batching statahead works correctly ========================================================== 10:56:24 (1789138584) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff97a0d11e4800.batch_stats=0 llite.lustre-ffff97a0d11e4800.statahead_max=32 llite.lustre-ffff97a0d11e4800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff97a0d11e4800.batch_stats=0 total: 100 open/close in 2.40 seconds: 41.61 ops/second llite.lustre-ffff97a0d11e4800.statahead_max=0 101 real 0m2.006s user 0m0.006s sys 0m0.301s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff97a0d11e4800.statahead_max=32 llite.lustre-ffff97a0d11e4800.statahead_max=32 101 real 0m1.099s user 0m0.005s sys 0m0.138s 'ls -l' 100 files with statahead: 1 sec statahead total: 488 statahead wrong: 1 agl total: 111 list_total: 30 fname_total: 458 hit_total: 4198 miss_total: 1298 swrong=1 ewrong=1 - open/close 751 (time 1789138612.26 total 10.01 last 75.03) total: 900 open/close in 11.98 seconds: 75.15 ops/second llite.lustre-ffff97a0d11e4800.statahead_max=0 1001 real 0m19.071s user 0m0.030s sys 0m2.848s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff97a0d11e4800.statahead_max=32 llite.lustre-ffff97a0d11e4800.statahead_max=32 1001 real 0m11.315s user 0m0.025s sys 0m1.336s 'ls -l' 1000 files with statahead: 12 sec statahead total: 489 statahead wrong: 1 agl total: 112 list_total: 31 fname_total: 458 hit_total: 5195 miss_total: 1301 swrong=1 ewrong=1 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff97a0d11e4800.statahead_stats= statahead total: 489 statahead wrong: 1 agl total: 112 list_total: 31 fname_total: 458 hit_total: 5195 miss_total: 1301 mdc.lustre-MDT0000-mdc-ffff97a0d11e4800.batch_stats= snapshot_time: 1789138677.616316738 secs.nsecs start_time: 1789138589.836429621 secs.nsecs elapsed_time: 87.779887117 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 829 (time 1789138689.90 total 10.01 last 82.83) - open/close 1624 (time 1789138699.90 total 20.01 last 79.47) - open/close 2406 (time 1789138709.91 total 30.02 last 78.14) - open/close 3182 (time 1789138719.93 total 40.04 last 77.43) - open/close 3972 (time 1789138729.94 total 50.05 last 78.94) - open/close 4826 (time 1789138739.94 total 60.05 last 85.38) - open/close 5677 (time 1789138749.95 total 70.06 last 84.99) - open/close 6496 (time 1789138759.97 total 80.08 last 81.78) - open/close 7337 (time 1789138769.98 total 90.09 last 84.02) - open/close 8105 (time 1789138779.98 total 100.09 last 76.79) - open/close 8831 (time 1789138789.99 total 110.10 last 72.55) - open/close 9544 (time 1789138799.99 total 120.10 last 71.26) total: 10000 open/close in 125.97 seconds: 79.38 ops/second llite.lustre-ffff97a0d11e4800.statahead_batch_max=0 llite.lustre-ffff97a0d11e4800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff97a0d11e4800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m3.490s user 0m0.318s sys 0m13.503s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff97a0d11e4800.statahead_batch_max=32 llite.lustre-ffff97a0d11e4800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff97a0d11e4800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff97a0d11e4800.stats=clear ls -l files in d123ad.sanity: 10001 real 2m1.498s user 0m0.318s sys 0m13.363s batched statahead RPC= hit count=9975 mdc.lustre-MDT0000-mdc-ffff97a0d11e4800.batch_stats= snapshot_time: 1789139012.001371314 secs.nsecs start_time: 1789138878.140290777 secs.nsecs elapsed_time: 133.861080537 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff97a0d11e4800.batch_stats=0 llite.lustre-ffff97a0d11e4800.statahead_max=2048 llite.lustre-ffff97a0d11e4800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff97a0d11e4800.batch_stats=0 total: 100 open/close in 3.35 seconds: 29.82 ops/second llite.lustre-ffff97a0d11e4800.statahead_max=0 101 real 0m3.472s user 0m0.006s sys 0m0.329s 'ls -l' 100 files without statahead: 4 sec llite.lustre-ffff97a0d11e4800.statahead_max=2048 llite.lustre-ffff97a0d11e4800.statahead_max=2048 101 real 0m1.612s user 0m0.005s sys 0m0.199s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10074 miss_total: 26 swrong=0 ewrong=0 - open/close 680 (time 1789139331.97 total 10.00 last 67.98) total: 900 open/close in 13.39 seconds: 67.21 ops/second llite.lustre-ffff97a0d11e4800.statahead_max=0 1001 real 0m25.471s user 0m0.030s sys 0m2.902s 'ls -l' 1000 files without statahead: 25 sec llite.lustre-ffff97a0d11e4800.statahead_max=2048 llite.lustre-ffff97a0d11e4800.statahead_max=2048 1001 real 0m7.907s user 0m0.026s sys 0m1.286s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11073 miss_total: 27 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 26 seconds llite.lustre-ffff97a0d11e4800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11073 miss_total: 27 mdc.lustre-MDT0000-mdc-ffff97a0d11e4800.batch_stats= snapshot_time: 1789139411.459830361 secs.nsecs start_time: 1789139015.079066173 secs.nsecs elapsed_time: 396.380764188 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 680 (time 1789139424.76 total 10.00 last 68.00) - open/close 1354 (time 1789139434.77 total 20.01 last 67.35) - open/close 2080 (time 1789139444.77 total 30.01 last 72.55) - open/close 2729 (time 1789139454.79 total 40.03 last 64.82) - open/close 3265 (time 1789139464.79 total 50.03 last 53.56) - open/close 3905 (time 1789139474.80 total 60.04 last 63.97) - open/close 4596 (time 1789139484.81 total 70.05 last 69.04) - open/close 5215 (time 1789139494.83 total 80.07 last 61.78) - open/close 5716 (time 1789139504.89 total 90.13 last 49.76) - open/close 6178 (time 1789139514.91 total 100.15 last 46.15) - open/close 6790 (time 1789139524.93 total 110.17 last 61.04) - open/close 7564 (time 1789139534.96 total 120.20 last 77.19) - open/close 8391 (time 1789139544.96 total 130.20 last 82.66) - open/close 9094 (time 1789139554.97 total 140.21 last 70.26) - open/close 9886 (time 1789139564.98 total 150.22 last 79.11) total: 10000 open/close in 151.41 seconds: 66.05 ops/second llite.lustre-ffff97a0d11e4800.statahead_batch_max=0 llite.lustre-ffff97a0d11e4800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff97a0d11e4800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m12.563s user 0m0.321s sys 0m13.544s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff97a0d11e4800.statahead_batch_max=256 llite.lustre-ffff97a0d11e4800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff97a0d11e4800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff97a0d11e4800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m56.790s user 0m0.301s sys 0m13.098s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff97a0d11e4800.batch_stats= snapshot_time: 1789139717.677459313 secs.nsecs start_time: 1789139647.978262347 secs.nsecs elapsed_time: 69.699196966 secs.nsecs subreqs per batch batches %% cum %% 1: 70 35 35 2: 31 15 50 4: 21 10 61 8: 7 3 64 16: 12 6 70 32: 16 8 78 64: 6 3 81 128: 2 1 82 256: 35 17 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff97a0d11e4800.statahead_batch_max=64 llite.lustre-ffff97a0d11e4800.statahead_max=128