== sanity test 123ad: Verify batching statahead works correctly ========================================================== 05:55:20 (1789293320) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9f63a9080000.batch_stats=0 llite.lustre-ffff9f63a9080000.statahead_max=32 llite.lustre-ffff9f63a9080000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9f63a9080000.batch_stats=0 total: 100 open/close in 2.45 seconds: 40.89 ops/second llite.lustre-ffff9f63a9080000.statahead_max=0 101 real 0m1.952s user 0m0.003s sys 0m0.291s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9f63a9080000.statahead_max=32 llite.lustre-ffff9f63a9080000.statahead_max=32 101 real 0m1.006s user 0m0.003s sys 0m0.159s 'ls -l' 100 files with statahead: 1 sec statahead total: 126 statahead wrong: 2 agl total: 121 list_total: 29 fname_total: 82 hit_total: 3578 miss_total: 1729 swrong=2 ewrong=2 - open/close 794 (time 1789293346.86 total 10.00 last 79.37) total: 900 open/close in 11.58 seconds: 77.69 ops/second llite.lustre-ffff9f63a9080000.statahead_max=0 1001 real 0m18.354s user 0m0.033s sys 0m2.861s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff9f63a9080000.statahead_max=32 llite.lustre-ffff9f63a9080000.statahead_max=32 1001 real 0m7.723s user 0m0.024s sys 0m1.225s 'ls -l' 1000 files with statahead: 8 sec statahead total: 127 statahead wrong: 2 agl total: 122 list_total: 30 fname_total: 82 hit_total: 4576 miss_total: 1731 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 18 seconds llite.lustre-ffff9f63a9080000.statahead_stats= statahead total: 127 statahead wrong: 2 agl total: 122 list_total: 30 fname_total: 82 hit_total: 4576 miss_total: 1731 mdc.lustre-MDT0000-mdc-ffff9f63a9080000.batch_stats= snapshot_time: 1789293404.111399768 secs.nsecs start_time: 1789293324.964345215 secs.nsecs elapsed_time: 79.147054553 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 764 (time 1789293416.50 total 10.01 last 76.33) - open/close 1647 (time 1789293426.51 total 20.02 last 88.20) - open/close 2548 (time 1789293436.53 total 30.03 last 89.97) - open/close 3300 (time 1789293446.54 total 40.05 last 75.11) - open/close 4140 (time 1789293456.54 total 50.05 last 84.00) - open/close 4832 (time 1789293466.55 total 60.05 last 69.15) - open/close 5612 (time 1789293476.55 total 70.06 last 77.99) - open/close 6315 (time 1789293486.56 total 80.07 last 70.23) - open/close 7034 (time 1789293496.56 total 90.07 last 71.87) - open/close 7689 (time 1789293506.57 total 100.08 last 65.45) - open/close 8396 (time 1789293516.58 total 110.09 last 70.63) - open/close 9079 (time 1789293526.58 total 120.09 last 68.30) - open/close 9908 (time 1789293536.58 total 130.09 last 82.85) total: 10000 open/close in 131.09 seconds: 76.28 ops/second llite.lustre-ffff9f63a9080000.statahead_batch_max=0 llite.lustre-ffff9f63a9080000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f63a9080000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m58.281s user 0m0.248s sys 0m12.823s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9f63a9080000.statahead_batch_max=32 llite.lustre-ffff9f63a9080000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f63a9080000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9f63a9080000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m26.798s user 0m0.277s sys 0m12.638s batched statahead RPC= hit count=9982 mdc.lustre-MDT0000-mdc-ffff9f63a9080000.batch_stats= snapshot_time: 1789293702.242126462 secs.nsecs start_time: 1789293604.971051740 secs.nsecs elapsed_time: 97.271074722 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9f63a9080000.batch_stats=0 llite.lustre-ffff9f63a9080000.statahead_max=2048 llite.lustre-ffff9f63a9080000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9f63a9080000.batch_stats=0 total: 100 open/close in 2.53 seconds: 39.60 ops/second llite.lustre-ffff9f63a9080000.statahead_max=0 101 real 0m2.702s user 0m0.005s sys 0m0.297s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9f63a9080000.statahead_max=2048 llite.lustre-ffff9f63a9080000.statahead_max=2048 101 real 0m0.920s user 0m0.003s sys 0m0.154s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10081 miss_total: 19 swrong=0 ewrong=0 - open/close 651 (time 1789294006.91 total 10.00 last 65.07) total: 900 open/close in 13.31 seconds: 67.60 ops/second llite.lustre-ffff9f63a9080000.statahead_max=0 1001 real 0m18.535s user 0m0.028s sys 0m2.815s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff9f63a9080000.statahead_max=2048 llite.lustre-ffff9f63a9080000.statahead_max=2048 1001 real 0m5.012s user 0m0.029s sys 0m1.450s 'ls -l' 1000 files with statahead: 5 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff9f63a9080000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 mdc.lustre-MDT0000-mdc-ffff9f63a9080000.batch_stats= snapshot_time: 1789294068.387667527 secs.nsecs start_time: 1789293704.778814873 secs.nsecs elapsed_time: 363.608852654 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 16 16 16: 2 16 33 32: 2 16 50 64: 1 8 58 128: 2 16 75 256: 3 25 100 - open/close 782 (time 1789294081.22 total 10.00 last 78.18) - open/close 1428 (time 1789294091.23 total 20.01 last 64.55) - open/close 2206 (time 1789294101.25 total 30.03 last 77.64) - open/close 2982 (time 1789294111.26 total 40.04 last 77.53) - open/close 3703 (time 1789294121.26 total 50.04 last 72.10) - open/close 4495 (time 1789294131.26 total 60.04 last 79.18) - open/close 5266 (time 1789294141.26 total 70.04 last 77.08) - open/close 6146 (time 1789294151.27 total 80.05 last 87.94) - open/close 6986 (time 1789294161.27 total 90.05 last 83.97) - open/close 7795 (time 1789294171.28 total 100.06 last 80.84) - open/close 8590 (time 1789294181.29 total 110.07 last 79.42) - open/close 9452 (time 1789294191.29 total 120.07 last 86.17) total: 10000 open/close in 125.86 seconds: 79.45 ops/second llite.lustre-ffff9f63a9080000.statahead_batch_max=0 llite.lustre-ffff9f63a9080000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f63a9080000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m50.131s user 0m0.282s sys 0m12.977s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9f63a9080000.statahead_batch_max=256 llite.lustre-ffff9f63a9080000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f63a9080000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9f63a9080000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m43.477s user 0m0.285s sys 0m13.215s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9f63a9080000.batch_stats= snapshot_time: 1789294307.833425110 secs.nsecs start_time: 1789294255.089929098 secs.nsecs elapsed_time: 52.743496012 secs.nsecs subreqs per batch batches %% cum %% 1: 81 36 36 2: 33 15 51 4: 21 9 61 8: 14 6 67 16: 13 5 73 32: 19 8 82 64: 4 1 84 128: 0 0 84 256: 35 15 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9f63a9080000.statahead_batch_max=64 llite.lustre-ffff9f63a9080000.statahead_max=128