== sanity test 123ad: Verify batching statahead works correctly ========================================================== 20:43:49 (1787445829) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9342875a5800.batch_stats=0 llite.lustre-ffff9342875a5800.statahead_max=32 llite.lustre-ffff9342875a5800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9342875a5800.batch_stats=0 total: 100 open/close in 3.10 seconds: 32.21 ops/second llite.lustre-ffff9342875a5800.statahead_max=0 101 real 0m2.769s user 0m0.009s sys 0m0.383s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9342875a5800.statahead_max=32 llite.lustre-ffff9342875a5800.statahead_max=32 101 real 0m1.245s user 0m0.005s sys 0m0.180s 'ls -l' 100 files with statahead: 1 sec statahead total: 120 statahead wrong: 2 agl total: 114 list_total: 29 fname_total: 91 hit_total: 4169 miss_total: 1275 swrong=2 ewrong=2 - open/close 716 (time 1787445859.10 total 10.00 last 71.59) total: 900 open/close in 13.11 seconds: 68.67 ops/second llite.lustre-ffff9342875a5800.statahead_max=0 1001 real 0m23.856s user 0m0.034s sys 0m3.324s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff9342875a5800.statahead_max=32 llite.lustre-ffff9342875a5800.statahead_max=32 1001 real 0m9.351s user 0m0.031s sys 0m1.398s 'ls -l' 1000 files with statahead: 9 sec statahead total: 121 statahead wrong: 2 agl total: 115 list_total: 30 fname_total: 91 hit_total: 5167 miss_total: 1277 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff9342875a5800.statahead_stats= statahead total: 121 statahead wrong: 2 agl total: 115 list_total: 30 fname_total: 91 hit_total: 5167 miss_total: 1277 mdc.lustre-MDT0000-mdc-ffff9342875a5800.batch_stats= snapshot_time: 1787445927.913987623 secs.nsecs start_time: 1787445835.069460527 secs.nsecs elapsed_time: 92.844527096 secs.nsecs subreqs per batch batches %% cum %% 1: 2 4 4 2: 1 2 7 4: 0 0 7 8: 2 4 12 16: 2 4 17 32: 34 82 100 - open/close 721 (time 1787445939.99 total 10.01 last 72.01) - open/close 1419 (time 1787445949.99 total 20.02 last 69.77) - open/close 2154 (time 1787445960.00 total 30.02 last 73.45) - open/close 2840 (time 1787445970.00 total 40.03 last 68.58) - open/close 3516 (time 1787445980.01 total 50.03 last 67.56) - open/close 4209 (time 1787445990.01 total 60.03 last 69.29) - open/close 4969 (time 1787446000.02 total 70.04 last 75.94) - open/close 5682 (time 1787446010.02 total 80.04 last 71.28) - open/close 6313 (time 1787446020.03 total 90.05 last 63.06) - open/close 6930 (time 1787446030.04 total 100.06 last 61.63) - open/close 7563 (time 1787446040.05 total 110.07 last 63.25) - open/close 8140 (time 1787446050.05 total 120.07 last 57.68) - open/close 8812 (time 1787446060.05 total 130.07 last 67.20) - open/close 9431 (time 1787446070.05 total 140.08 last 61.88) total: 10000 open/close in 148.85 seconds: 67.18 ops/second llite.lustre-ffff9342875a5800.statahead_batch_max=0 llite.lustre-ffff9342875a5800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9342875a5800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m56.880s user 0m0.312s sys 0m15.223s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9342875a5800.statahead_batch_max=32 llite.lustre-ffff9342875a5800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9342875a5800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9342875a5800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m46.969s user 0m0.279s sys 0m13.550s batched statahead RPC= hit count=9973 mdc.lustre-MDT0000-mdc-ffff9342875a5800.batch_stats= snapshot_time: 1787446259.708695750 secs.nsecs start_time: 1787446142.821733170 secs.nsecs elapsed_time: 116.886962580 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9342875a5800.batch_stats=0 llite.lustre-ffff9342875a5800.statahead_max=2048 llite.lustre-ffff9342875a5800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9342875a5800.batch_stats=0 total: 100 open/close in 2.74 seconds: 36.47 ops/second llite.lustre-ffff9342875a5800.statahead_max=0 101 real 0m2.382s user 0m0.008s sys 0m0.343s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9342875a5800.statahead_max=2048 llite.lustre-ffff9342875a5800.statahead_max=2048 101 real 0m0.697s user 0m0.007s sys 0m0.167s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10072 miss_total: 28 swrong=0 ewrong=0 - open/close 671 (time 1787446602.12 total 10.01 last 67.00) total: 900 open/close in 13.38 seconds: 67.27 ops/second llite.lustre-ffff9342875a5800.statahead_max=0 1001 real 0m20.643s user 0m0.040s sys 0m2.874s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff9342875a5800.statahead_max=2048 llite.lustre-ffff9342875a5800.statahead_max=2048 1001 real 0m6.837s user 0m0.026s sys 0m1.381s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11071 miss_total: 29 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 29 seconds llite.lustre-ffff9342875a5800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11071 miss_total: 29 mdc.lustre-MDT0000-mdc-ffff9342875a5800.batch_stats= snapshot_time: 1787446674.096469791 secs.nsecs start_time: 1787446262.283346179 secs.nsecs elapsed_time: 411.813123612 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 669 (time 1787446686.94 total 10.01 last 66.86) - open/close 1197 (time 1787446696.95 total 20.01 last 52.76) - open/close 1791 (time 1787446706.95 total 30.02 last 59.37) - open/close 2376 (time 1787446716.98 total 40.05 last 58.31) - open/close 3009 (time 1787446726.99 total 50.06 last 63.26) - open/close 3714 (time 1787446736.99 total 60.06 last 70.49) - open/close 4433 (time 1787446747.00 total 70.07 last 71.85) - open/close 5132 (time 1787446757.00 total 80.07 last 69.90) - open/close 5820 (time 1787446767.00 total 90.07 last 68.79) - open/close 6560 (time 1787446777.00 total 100.07 last 73.99) - open/close 7310 (time 1787446787.01 total 110.07 last 74.97) - open/close 7957 (time 1787446797.01 total 120.08 last 64.69) - open/close 8638 (time 1787446807.01 total 130.08 last 68.07) - open/close 9294 (time 1787446817.03 total 140.10 last 65.50) - open/close 9940 (time 1787446827.04 total 150.11 last 64.52) total: 10000 open/close in 151.08 seconds: 66.19 ops/second llite.lustre-ffff9342875a5800.statahead_batch_max=0 llite.lustre-ffff9342875a5800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9342875a5800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m9.407s user 0m0.305s sys 0m14.898s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9342875a5800.statahead_batch_max=256 llite.lustre-ffff9342875a5800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9342875a5800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9342875a5800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m56.356s user 0m0.249s sys 0m13.776s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9342875a5800.batch_stats= snapshot_time: 1787446974.855183491 secs.nsecs start_time: 1787446907.059812210 secs.nsecs elapsed_time: 67.795371281 secs.nsecs subreqs per batch batches %% cum %% 1: 81 37 37 2: 28 13 50 4: 21 9 60 8: 17 7 68 16: 15 7 75 32: 15 7 82 64: 1 0 83 128: 0 0 83 256: 36 16 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9342875a5800.statahead_batch_max=64 llite.lustre-ffff9342875a5800.statahead_max=128