== sanity test 123ad: Verify batching statahead works correctly ========================================================== 02:24:20 (1789021460) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff91ff0964e000.batch_stats=0 llite.lustre-ffff91ff0964e000.statahead_max=32 llite.lustre-ffff91ff0964e000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff91ff0964e000.batch_stats=0 total: 100 open/close in 2.51 seconds: 39.81 ops/second llite.lustre-ffff91ff0964e000.statahead_max=0 101 real 0m2.573s user 0m0.003s sys 0m0.303s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff91ff0964e000.statahead_max=32 llite.lustre-ffff91ff0964e000.statahead_max=32 101 real 0m1.212s user 0m0.006s sys 0m0.178s 'ls -l' 100 files with statahead: 1 sec statahead total: 99 statahead wrong: 2 agl total: 94 list_total: 30 fname_total: 69 hit_total: 4442 miss_total: 1285 swrong=2 ewrong=2 - open/close 814 (time 1789021487.82 total 10.01 last 81.36) total: 900 open/close in 11.27 seconds: 79.86 ops/second llite.lustre-ffff91ff0964e000.statahead_max=0 1001 real 0m20.108s user 0m0.030s sys 0m2.790s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff91ff0964e000.statahead_max=32 llite.lustre-ffff91ff0964e000.statahead_max=32 1001 real 0m9.547s user 0m0.032s sys 0m1.361s 'ls -l' 1000 files with statahead: 9 sec statahead total: 100 statahead wrong: 2 agl total: 95 list_total: 31 fname_total: 69 hit_total: 5439 miss_total: 1288 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff91ff0964e000.statahead_stats= statahead total: 100 statahead wrong: 2 agl total: 95 list_total: 31 fname_total: 69 hit_total: 5439 miss_total: 1288 mdc.lustre-MDT0000-mdc-ffff91ff0964e000.batch_stats= snapshot_time: 1789021556.090891854 secs.nsecs start_time: 1789021464.834025444 secs.nsecs elapsed_time: 91.256866410 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 656 (time 1789021568.84 total 10.00 last 65.57) - open/close 1273 (time 1789021578.85 total 20.00 last 61.70) - open/close 1919 (time 1789021588.85 total 30.01 last 64.57) - open/close 2557 (time 1789021598.85 total 40.01 last 63.80) - open/close 3318 (time 1789021608.86 total 50.02 last 76.04) - open/close 3999 (time 1789021618.86 total 60.02 last 68.07) - open/close 4673 (time 1789021628.87 total 70.03 last 67.36) - open/close 5322 (time 1789021638.87 total 80.03 last 64.86) - open/close 6105 (time 1789021648.88 total 90.04 last 78.23) - open/close 6850 (time 1789021658.89 total 100.05 last 74.43) - open/close 7691 (time 1789021668.90 total 110.06 last 84.02) - open/close 8459 (time 1789021678.91 total 120.07 last 76.75) - open/close 9144 (time 1789021688.92 total 130.08 last 68.45) - open/close 9834 (time 1789021698.92 total 140.08 last 68.98) total: 10000 open/close in 142.21 seconds: 70.32 ops/second llite.lustre-ffff91ff0964e000.statahead_batch_max=0 llite.lustre-ffff91ff0964e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff91ff0964e000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m15.608s user 0m0.324s sys 0m14.637s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff91ff0964e000.statahead_batch_max=32 llite.lustre-ffff91ff0964e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff91ff0964e000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff91ff0964e000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m36.838s user 0m0.269s sys 0m13.202s batched statahead RPC= hit count=9988 mdc.lustre-MDT0000-mdc-ffff91ff0964e000.batch_stats= snapshot_time: 1789021892.999086707 secs.nsecs start_time: 1789021787.434576490 secs.nsecs elapsed_time: 105.564510217 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff91ff0964e000.batch_stats=0 llite.lustre-ffff91ff0964e000.statahead_max=2048 llite.lustre-ffff91ff0964e000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff91ff0964e000.batch_stats=0 total: 100 open/close in 2.94 seconds: 34.01 ops/second llite.lustre-ffff91ff0964e000.statahead_max=0 101 real 0m2.400s user 0m0.005s sys 0m0.292s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff91ff0964e000.statahead_max=2048 llite.lustre-ffff91ff0964e000.statahead_max=2048 101 real 0m1.070s user 0m0.001s sys 0m0.137s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10087 miss_total: 13 swrong=0 ewrong=0 - open/close 825 (time 1789022195.33 total 10.01 last 82.45) total: 900 open/close in 10.93 seconds: 82.38 ops/second llite.lustre-ffff91ff0964e000.statahead_max=0 1001 real 0m18.155s user 0m0.029s sys 0m2.693s 'ls -l' 1000 files without statahead: 18 sec llite.lustre-ffff91ff0964e000.statahead_max=2048 llite.lustre-ffff91ff0964e000.statahead_max=2048 1001 real 0m5.717s user 0m0.019s sys 0m1.377s 'ls -l' 1000 files with statahead: 5 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11086 miss_total: 14 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff91ff0964e000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11086 miss_total: 14 mdc.lustre-MDT0000-mdc-ffff91ff0964e000.batch_stats= snapshot_time: 1789022249.177871583 secs.nsecs start_time: 1789021895.797528825 secs.nsecs elapsed_time: 353.380342758 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 0 0 40 64: 1 10 50 128: 1 10 60 256: 4 40 100 - open/close 804 (time 1789022261.58 total 10.00 last 80.39) - open/close 1647 (time 1789022271.59 total 20.00 last 84.28) - open/close 2485 (time 1789022281.60 total 30.01 last 83.73) - open/close 3280 (time 1789022291.60 total 40.02 last 79.47) - open/close 4137 (time 1789022301.61 total 50.03 last 85.62) - open/close 4936 (time 1789022311.61 total 60.03 last 79.85) - open/close 5672 (time 1789022321.62 total 70.03 last 73.57) - open/close 6506 (time 1789022331.62 total 80.04 last 83.38) - open/close 7282 (time 1789022341.62 total 90.04 last 77.60) - open/close 8106 (time 1789022351.62 total 100.04 last 82.38) - open/close 8928 (time 1789022361.64 total 110.06 last 82.05) - open/close 9709 (time 1789022371.64 total 120.06 last 78.07) total: 10000 open/close in 123.77 seconds: 80.80 ops/second llite.lustre-ffff91ff0964e000.statahead_batch_max=0 llite.lustre-ffff91ff0964e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff91ff0964e000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m49.754s user 0m0.282s sys 0m15.957s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff91ff0964e000.statahead_batch_max=256 llite.lustre-ffff91ff0964e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff91ff0964e000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff91ff0964e000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m43.315s user 0m0.278s sys 0m15.060s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff91ff0964e000.batch_stats= snapshot_time: 1789022485.245493473 secs.nsecs start_time: 1789022432.857384511 secs.nsecs elapsed_time: 52.388108962 secs.nsecs subreqs per batch batches %% cum %% 1: 101 37 37 2: 40 14 52 4: 36 13 65 8: 24 8 74 16: 11 4 78 32: 12 4 83 64: 9 3 86 128: 2 0 87 256: 34 12 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff91ff0964e000.statahead_batch_max=64 llite.lustre-ffff91ff0964e000.statahead_max=128