== sanity test 123ad: Verify batching statahead works correctly ========================================================== 05:21:40 (1788945700) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9c20be31d000.batch_stats=0 llite.lustre-ffff9c20be31d000.statahead_max=32 llite.lustre-ffff9c20be31d000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9c20be31d000.batch_stats=0 total: 100 open/close in 3.00 seconds: 33.33 ops/second llite.lustre-ffff9c20be31d000.statahead_max=0 101 real 0m2.107s user 0m0.004s sys 0m0.285s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9c20be31d000.statahead_max=32 llite.lustre-ffff9c20be31d000.statahead_max=32 101 real 0m0.976s user 0m0.004s sys 0m0.157s 'ls -l' 100 files with statahead: 1 sec statahead total: 93 statahead wrong: 2 agl total: 88 list_total: 30 fname_total: 62 hit_total: 3892 miss_total: 1706 swrong=2 ewrong=2 - open/close 643 (time 1788945728.10 total 10.00 last 64.27) total: 900 open/close in 13.49 seconds: 66.71 ops/second llite.lustre-ffff9c20be31d000.statahead_max=0 1001 real 0m21.742s user 0m0.034s sys 0m2.730s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff9c20be31d000.statahead_max=32 llite.lustre-ffff9c20be31d000.statahead_max=32 1001 real 0m8.924s user 0m0.025s sys 0m1.336s 'ls -l' 1000 files with statahead: 9 sec statahead total: 94 statahead wrong: 2 agl total: 89 list_total: 31 fname_total: 62 hit_total: 4888 miss_total: 1710 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff9c20be31d000.statahead_stats= statahead total: 94 statahead wrong: 2 agl total: 89 list_total: 31 fname_total: 62 hit_total: 4888 miss_total: 1710 mdc.lustre-MDT0000-mdc-ffff9c20be31d000.batch_stats= snapshot_time: 1788945794.859972490 secs.nsecs start_time: 1788945704.317787013 secs.nsecs elapsed_time: 90.542185477 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 847 (time 1788945807.43 total 10.01 last 84.63) - open/close 1671 (time 1788945817.44 total 20.01 last 82.39) - open/close 2498 (time 1788945827.44 total 30.01 last 82.67) - open/close 3306 (time 1788945837.44 total 40.02 last 80.78) - open/close 4197 (time 1788945847.46 total 50.03 last 88.95) - open/close 5035 (time 1788945857.47 total 60.04 last 83.73) - open/close 5859 (time 1788945867.48 total 70.06 last 82.28) - open/close 6655 (time 1788945877.50 total 80.07 last 79.50) - open/close 7515 (time 1788945887.50 total 90.08 last 85.94) - open/close 8343 (time 1788945897.50 total 100.08 last 82.79) - open/close 9135 (time 1788945907.51 total 110.08 last 79.15) - open/close 9861 (time 1788945917.51 total 120.08 last 72.60) total: 10000 open/close in 121.59 seconds: 82.24 ops/second llite.lustre-ffff9c20be31d000.statahead_batch_max=0 llite.lustre-ffff9c20be31d000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9c20be31d000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m0.482s user 0m0.292s sys 0m14.355s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9c20be31d000.statahead_batch_max=32 llite.lustre-ffff9c20be31d000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9c20be31d000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9c20be31d000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m27.081s user 0m0.283s sys 0m13.382s batched statahead RPC= hit count=9985 mdc.lustre-MDT0000-mdc-ffff9c20be31d000.batch_stats= snapshot_time: 1788946082.813411111 secs.nsecs start_time: 1788945986.146547121 secs.nsecs elapsed_time: 96.666863990 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9c20be31d000.batch_stats=0 llite.lustre-ffff9c20be31d000.statahead_max=2048 llite.lustre-ffff9c20be31d000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9c20be31d000.batch_stats=0 total: 100 open/close in 2.58 seconds: 38.74 ops/second llite.lustre-ffff9c20be31d000.statahead_max=0 101 real 0m1.902s user 0m0.004s sys 0m0.286s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9c20be31d000.statahead_max=2048 llite.lustre-ffff9c20be31d000.statahead_max=2048 101 real 0m0.965s user 0m0.005s sys 0m0.170s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10084 miss_total: 16 swrong=0 ewrong=0 - open/close 893 (time 1788946372.37 total 10.01 last 89.21) total: 900 open/close in 10.10 seconds: 89.11 ops/second llite.lustre-ffff9c20be31d000.statahead_max=0 1001 real 0m21.128s user 0m0.034s sys 0m2.788s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff9c20be31d000.statahead_max=2048 llite.lustre-ffff9c20be31d000.statahead_max=2048 1001 real 0m5.526s user 0m0.020s sys 0m1.635s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11083 miss_total: 17 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff9c20be31d000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11083 miss_total: 17 mdc.lustre-MDT0000-mdc-ffff9c20be31d000.batch_stats= snapshot_time: 1788946430.838879939 secs.nsecs start_time: 1788946084.993232567 secs.nsecs elapsed_time: 345.845647372 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 804 (time 1788946443.21 total 10.02 last 80.23) - open/close 1600 (time 1788946453.21 total 20.02 last 79.58) - open/close 2420 (time 1788946463.21 total 30.03 last 81.97) - open/close 3207 (time 1788946473.21 total 40.03 last 78.70) - open/close 4035 (time 1788946483.22 total 50.04 last 82.73) - open/close 4788 (time 1788946493.23 total 60.05 last 75.21) - open/close 5554 (time 1788946503.25 total 70.07 last 76.46) - open/close 6313 (time 1788946513.27 total 80.08 last 75.81) - open/close 7084 (time 1788946523.27 total 90.08 last 77.10) - open/close 7819 (time 1788946533.27 total 100.08 last 73.49) - open/close 8525 (time 1788946543.28 total 110.10 last 70.49) - open/close 9256 (time 1788946553.30 total 120.12 last 72.95) total: 10000 open/close in 129.21 seconds: 77.40 ops/second llite.lustre-ffff9c20be31d000.statahead_batch_max=0 llite.lustre-ffff9c20be31d000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9c20be31d000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m2.064s user 0m0.301s sys 0m14.612s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9c20be31d000.statahead_batch_max=256 llite.lustre-ffff9c20be31d000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9c20be31d000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9c20be31d000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m45.299s user 0m0.260s sys 0m13.607s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9c20be31d000.batch_stats= snapshot_time: 1788946689.359489278 secs.nsecs start_time: 1788946633.519765588 secs.nsecs elapsed_time: 55.839723690 secs.nsecs subreqs per batch batches %% cum %% 1: 47 33 33 2: 12 8 42 4: 5 3 45 8: 9 6 52 16: 8 5 57 32: 18 12 70 64: 4 2 73 128: 1 0 74 256: 36 25 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9c20be31d000.statahead_batch_max=64 llite.lustre-ffff9c20be31d000.statahead_max=128