== sanity test 123ad: Verify batching statahead works correctly ========================================================== 17:41:12 (1787607672) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff934a84777000.batch_stats=0 llite.lustre-ffff934a84777000.statahead_max=32 llite.lustre-ffff934a84777000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff934a84777000.batch_stats=0 total: 100 open/close in 3.14 seconds: 31.83 ops/second llite.lustre-ffff934a84777000.statahead_max=0 101 real 0m2.424s user 0m0.007s sys 0m0.324s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff934a84777000.statahead_max=32 llite.lustre-ffff934a84777000.statahead_max=32 101 real 0m1.010s user 0m0.008s sys 0m0.177s 'ls -l' 100 files with statahead: 1 sec statahead total: 122 statahead wrong: 2 agl total: 116 list_total: 30 fname_total: 77 hit_total: 4033 miss_total: 1581 swrong=2 ewrong=2 - open/close 572 (time 1787607699.93 total 10.00 last 57.20) total: 900 open/close in 15.60 seconds: 57.70 ops/second llite.lustre-ffff934a84777000.statahead_max=0 1001 real 0m24.368s user 0m0.033s sys 0m3.098s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff934a84777000.statahead_max=32 llite.lustre-ffff934a84777000.statahead_max=32 1001 real 0m10.980s user 0m0.025s sys 0m1.496s 'ls -l' 1000 files with statahead: 11 sec statahead total: 123 statahead wrong: 2 agl total: 117 list_total: 31 fname_total: 77 hit_total: 5031 miss_total: 1583 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff934a84777000.statahead_stats= statahead total: 123 statahead wrong: 2 agl total: 117 list_total: 31 fname_total: 77 hit_total: 5031 miss_total: 1583 mdc.lustre-MDT0000-mdc-ffff934a84777000.batch_stats= snapshot_time: 1787607776.388508044 secs.nsecs start_time: 1787607677.107808075 secs.nsecs elapsed_time: 99.280699969 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 656 (time 1787607789.20 total 10.01 last 65.56) - open/close 1244 (time 1787607799.21 total 20.01 last 58.79) - open/close 1825 (time 1787607809.23 total 30.03 last 57.98) - open/close 2477 (time 1787607819.25 total 40.05 last 65.03) - open/close 3126 (time 1787607829.26 total 50.06 last 64.83) - open/close 3743 (time 1787607839.27 total 60.07 last 61.67) - open/close 4293 (time 1787607849.28 total 70.08 last 54.93) - open/close 4863 (time 1787607859.28 total 80.09 last 56.98) - open/close 5398 (time 1787607869.29 total 90.09 last 53.49) - open/close 5943 (time 1787607879.29 total 100.09 last 54.47) - open/close 6437 (time 1787607889.30 total 110.10 last 49.36) - open/close 6883 (time 1787607899.33 total 120.14 last 44.44) - open/close 7422 (time 1787607909.34 total 130.14 last 53.89) - open/close 7960 (time 1787607919.35 total 140.15 last 53.74) - open/close 8486 (time 1787607929.37 total 150.17 last 52.48) - open/close 9101 (time 1787607939.39 total 160.19 last 61.38) - open/close 9726 (time 1787607949.41 total 170.21 last 62.35) total: 10000 open/close in 175.20 seconds: 57.08 ops/second llite.lustre-ffff934a84777000.statahead_batch_max=0 llite.lustre-ffff934a84777000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff934a84777000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m11.847s user 0m0.406s sys 0m17.130s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff934a84777000.statahead_batch_max=32 llite.lustre-ffff934a84777000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff934a84777000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff934a84777000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m56.948s user 0m0.371s sys 0m15.512s batched statahead RPC= hit count=9978 mdc.lustre-MDT0000-mdc-ffff934a84777000.batch_stats= snapshot_time: 1787608165.148367884 secs.nsecs start_time: 1787608037.003916646 secs.nsecs elapsed_time: 128.144451238 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff934a84777000.batch_stats=0 llite.lustre-ffff934a84777000.statahead_max=2048 llite.lustre-ffff934a84777000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff934a84777000.batch_stats=0 total: 100 open/close in 3.32 seconds: 30.10 ops/second llite.lustre-ffff934a84777000.statahead_max=0 101 real 0m3.057s user 0m0.007s sys 0m0.342s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff934a84777000.statahead_max=2048 llite.lustre-ffff934a84777000.statahead_max=2048 101 real 0m1.162s user 0m0.005s sys 0m0.183s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10077 miss_total: 23 swrong=0 ewrong=0 - open/close 444 (time 1787608540.28 total 10.00 last 44.39) total: 900 open/close in 18.72 seconds: 48.07 ops/second llite.lustre-ffff934a84777000.statahead_max=0 1001 real 0m28.259s user 0m0.049s sys 0m3.688s 'ls -l' 1000 files without statahead: 28 sec llite.lustre-ffff934a84777000.statahead_max=2048 llite.lustre-ffff934a84777000.statahead_max=2048 1001 real 0m7.317s user 0m0.035s sys 0m1.617s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11076 miss_total: 24 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff934a84777000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11076 miss_total: 24 mdc.lustre-MDT0000-mdc-ffff934a84777000.batch_stats= snapshot_time: 1787608623.098350831 secs.nsecs start_time: 1787608167.935505569 secs.nsecs elapsed_time: 455.162845262 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 16 16 16: 2 16 33 32: 2 16 50 64: 1 8 58 128: 1 8 66 256: 4 33 100 - open/close 602 (time 1787608636.28 total 10.01 last 60.14) - open/close 1187 (time 1787608646.28 total 20.01 last 58.50) - open/close 1807 (time 1787608656.31 total 30.04 last 61.81) - open/close 2360 (time 1787608666.32 total 40.05 last 55.27) - open/close 2945 (time 1787608676.32 total 50.05 last 58.50) - open/close 3476 (time 1787608686.32 total 60.06 last 53.05) - open/close 4037 (time 1787608696.33 total 70.06 last 56.06) - open/close 4494 (time 1787608706.34 total 80.07 last 45.66) - open/close 5075 (time 1787608716.34 total 90.07 last 58.09) - open/close 5702 (time 1787608726.35 total 100.08 last 62.63) - open/close 6261 (time 1787608736.37 total 110.10 last 55.82) - open/close 6794 (time 1787608746.38 total 120.11 last 53.24) - open/close 7394 (time 1787608756.39 total 130.12 last 59.92) - open/close 8055 (time 1787608766.40 total 140.14 last 66.03) - open/close 8726 (time 1787608776.41 total 150.14 last 67.08) - open/close 9368 (time 1787608786.42 total 160.15 last 64.13) - open/close 9932 (time 1787608796.43 total 170.16 last 56.33) total: 10000 open/close in 171.25 seconds: 58.39 ops/second llite.lustre-ffff934a84777000.statahead_batch_max=0 llite.lustre-ffff934a84777000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff934a84777000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m4.250s user 0m0.390s sys 0m16.245s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff934a84777000.statahead_batch_max=256 llite.lustre-ffff934a84777000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff934a84777000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff934a84777000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m49.777s user 0m0.341s sys 0m16.580s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff934a84777000.batch_stats= snapshot_time: 1787608933.436382869 secs.nsecs start_time: 1787608871.867275612 secs.nsecs elapsed_time: 61.569107257 secs.nsecs subreqs per batch batches %% cum %% 1: 146 51 51 2: 37 12 63 4: 27 9 73 8: 13 4 77 16: 8 2 80 32: 11 3 84 64: 8 2 87 128: 2 0 88 256: 34 11 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff934a84777000.statahead_batch_max=64 llite.lustre-ffff934a84777000.statahead_max=128