== sanity test 123ad: Verify batching statahead works correctly ========================================================== 09:34:39 (1787837679) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8b23923de000.batch_stats=0 llite.lustre-ffff8b23923de000.statahead_max=32 llite.lustre-ffff8b23923de000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8b23923de000.batch_stats=0 total: 100 open/close in 4.15 seconds: 24.11 ops/second llite.lustre-ffff8b23923de000.statahead_max=0 101 real 0m3.682s user 0m0.007s sys 0m0.376s 'ls -l' 100 files without statahead: 4 sec llite.lustre-ffff8b23923de000.statahead_max=32 llite.lustre-ffff8b23923de000.statahead_max=32 101 real 0m1.278s user 0m0.003s sys 0m0.167s 'ls -l' 100 files with statahead: 2 sec statahead total: 117 statahead wrong: 2 agl total: 111 list_total: 30 fname_total: 85 hit_total: 3604 miss_total: 1590 swrong=2 ewrong=2 - open/close 524 (time 1787837713.16 total 10.01 last 52.34) total: 900 open/close in 17.01 seconds: 52.90 ops/second llite.lustre-ffff8b23923de000.statahead_max=0 1001 real 0m27.097s user 0m0.036s sys 0m3.052s 'ls -l' 1000 files without statahead: 27 sec llite.lustre-ffff8b23923de000.statahead_max=32 llite.lustre-ffff8b23923de000.statahead_max=32 1001 real 0m10.157s user 0m0.028s sys 0m1.464s 'ls -l' 1000 files with statahead: 10 sec statahead total: 118 statahead wrong: 2 agl total: 112 list_total: 31 fname_total: 85 hit_total: 4599 miss_total: 1595 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 27 seconds llite.lustre-ffff8b23923de000.statahead_stats= statahead total: 118 statahead wrong: 2 agl total: 112 list_total: 31 fname_total: 85 hit_total: 4599 miss_total: 1595 mdc.lustre-MDT0000-mdc-ffff8b23923de000.batch_stats= snapshot_time: 1787837798.880620752 secs.nsecs start_time: 1787837684.430629853 secs.nsecs elapsed_time: 114.449990899 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 523 (time 1787837812.38 total 10.01 last 52.27) - open/close 1052 (time 1787837822.39 total 20.02 last 52.85) - open/close 1683 (time 1787837832.40 total 30.03 last 63.03) - open/close 2138 (time 1787837842.41 total 40.03 last 45.47) - open/close 2675 (time 1787837852.42 total 50.05 last 53.62) - open/close 3185 (time 1787837862.42 total 60.05 last 50.98) - open/close 3658 (time 1787837872.45 total 70.07 last 47.19) - open/close 4158 (time 1787837882.48 total 80.10 last 49.85) - open/close 4703 (time 1787837892.48 total 90.11 last 54.46) - open/close 5245 (time 1787837902.49 total 100.11 last 54.19) - open/close 5798 (time 1787837912.50 total 110.12 last 55.25) - open/close 6332 (time 1787837922.55 total 120.18 last 53.09) - open/close 6908 (time 1787837932.55 total 130.18 last 57.60) - open/close 7474 (time 1787837942.55 total 140.18 last 56.60) - open/close 7988 (time 1787837952.57 total 150.19 last 51.33) - open/close 8408 (time 1787837962.58 total 160.20 last 41.96) - open/close 8927 (time 1787837972.58 total 170.21 last 51.89) - open/close 9454 (time 1787837982.59 total 180.22 last 52.64) total: 10000 open/close in 189.94 seconds: 52.65 ops/second llite.lustre-ffff8b23923de000.statahead_batch_max=0 llite.lustre-ffff8b23923de000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b23923de000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m13.752s user 0m0.338s sys 0m14.903s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8b23923de000.statahead_batch_max=32 llite.lustre-ffff8b23923de000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b23923de000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8b23923de000.stats=clear ls -l files in d123ad.sanity: 10001 real 2m2.592s user 0m0.359s sys 0m14.322s batched statahead RPC= hit count=9977 mdc.lustre-MDT0000-mdc-ffff8b23923de000.batch_stats= snapshot_time: 1787838209.414331997 secs.nsecs start_time: 1787838075.281945463 secs.nsecs elapsed_time: 134.132386534 secs.nsecs subreqs per batch batches %% cum %% 1: 1 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8b23923de000.batch_stats=0 llite.lustre-ffff8b23923de000.statahead_max=2048 llite.lustre-ffff8b23923de000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8b23923de000.batch_stats=0 total: 100 open/close in 3.31 seconds: 30.19 ops/second llite.lustre-ffff8b23923de000.statahead_max=0 101 real 0m2.402s user 0m0.004s sys 0m0.322s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8b23923de000.statahead_max=2048 llite.lustre-ffff8b23923de000.statahead_max=2048 101 real 0m1.178s user 0m0.009s sys 0m0.173s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10076 miss_total: 24 swrong=0 ewrong=0 - open/close 524 (time 1787838588.21 total 10.01 last 52.36) total: 900 open/close in 16.85 seconds: 53.41 ops/second llite.lustre-ffff8b23923de000.statahead_max=0 1001 real 0m27.010s user 0m0.041s sys 0m2.978s 'ls -l' 1000 files without statahead: 27 sec llite.lustre-ffff8b23923de000.statahead_max=2048 llite.lustre-ffff8b23923de000.statahead_max=2048 1001 real 0m5.637s user 0m0.027s sys 0m1.439s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11075 miss_total: 25 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff8b23923de000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11075 miss_total: 25 mdc.lustre-MDT0000-mdc-ffff8b23923de000.batch_stats= snapshot_time: 1787838665.371670662 secs.nsecs start_time: 1787838212.726029046 secs.nsecs elapsed_time: 452.645641616 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 1 9 63 128: 0 0 63 256: 4 36 100 - open/close 645 (time 1787838677.84 total 10.00 last 64.47) - open/close 1268 (time 1787838687.85 total 20.02 last 62.23) - open/close 1892 (time 1787838697.86 total 30.02 last 62.35) - open/close 2587 (time 1787838707.86 total 40.03 last 69.46) - open/close 3251 (time 1787838717.89 total 50.06 last 66.23) - open/close 3808 (time 1787838727.90 total 60.07 last 55.62) - open/close 4440 (time 1787838737.91 total 70.08 last 63.16) - open/close 5160 (time 1787838747.92 total 80.08 last 71.95) - open/close 5791 (time 1787838757.93 total 90.09 last 63.05) - open/close 6409 (time 1787838767.93 total 100.10 last 61.75) - open/close 7089 (time 1787838777.94 total 110.10 last 67.97) - open/close 7746 (time 1787838787.95 total 120.12 last 65.60) - open/close 8449 (time 1787838797.96 total 130.13 last 70.24) - open/close 9103 (time 1787838807.97 total 140.14 last 65.33) - open/close 9964 (time 1787838817.97 total 150.14 last 86.07) total: 10000 open/close in 150.51 seconds: 66.44 ops/second llite.lustre-ffff8b23923de000.statahead_batch_max=0 llite.lustre-ffff8b23923de000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b23923de000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m59.602s user 0m0.314s sys 0m14.068s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8b23923de000.statahead_batch_max=256 llite.lustre-ffff8b23923de000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b23923de000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8b23923de000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m50.731s user 0m0.271s sys 0m14.363s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8b23923de000.batch_stats= snapshot_time: 1787838949.343435505 secs.nsecs start_time: 1787838888.232404184 secs.nsecs elapsed_time: 61.111031321 secs.nsecs subreqs per batch batches %% cum %% 1: 48 33 33 2: 16 11 44 4: 8 5 50 8: 5 3 53 16: 12 8 62 32: 15 10 72 64: 3 2 74 128: 0 0 74 256: 36 25 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8b23923de000.statahead_batch_max=64 llite.lustre-ffff8b23923de000.statahead_max=128