== sanity test 123ad: Verify batching statahead works correctly ========================================================== 06:34:09 (1787740449) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8b2274ddb800.batch_stats=0 llite.lustre-ffff8b2274ddb800.statahead_max=32 llite.lustre-ffff8b2274ddb800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8b2274ddb800.batch_stats=0 total: 100 open/close in 3.30 seconds: 30.29 ops/second llite.lustre-ffff8b2274ddb800.statahead_max=0 101 real 0m2.513s user 0m0.005s sys 0m0.316s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8b2274ddb800.statahead_max=32 llite.lustre-ffff8b2274ddb800.statahead_max=32 101 real 0m0.897s user 0m0.004s sys 0m0.188s 'ls -l' 100 files with statahead: 1 sec statahead total: 118 statahead wrong: 2 agl total: 112 list_total: 31 fname_total: 66 hit_total: 3869 miss_total: 1684 swrong=2 ewrong=2 - open/close 614 (time 1787740475.49 total 10.02 last 61.29) total: 900 open/close in 14.80 seconds: 60.82 ops/second llite.lustre-ffff8b2274ddb800.statahead_max=0 1001 real 0m23.554s user 0m0.037s sys 0m2.979s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff8b2274ddb800.statahead_max=32 llite.lustre-ffff8b2274ddb800.statahead_max=32 1001 real 0m14.303s user 0m0.038s sys 0m1.538s 'ls -l' 1000 files with statahead: 14 sec statahead total: 119 statahead wrong: 2 agl total: 113 list_total: 32 fname_total: 66 hit_total: 4863 miss_total: 1690 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 26 seconds llite.lustre-ffff8b2274ddb800.statahead_stats= statahead total: 119 statahead wrong: 2 agl total: 113 list_total: 32 fname_total: 66 hit_total: 4863 miss_total: 1690 mdc.lustre-MDT0000-mdc-ffff8b2274ddb800.batch_stats= snapshot_time: 1787740558.455122441 secs.nsecs start_time: 1787740453.748347727 secs.nsecs elapsed_time: 104.706774714 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 2 5 7 16: 3 7 15 32: 34 85 100 - open/close 465 (time 1787740571.56 total 10.02 last 46.40) - open/close 1078 (time 1787740581.58 total 20.04 last 61.18) - open/close 1829 (time 1787740591.59 total 30.05 last 75.06) - open/close 2684 (time 1787740601.59 total 40.05 last 85.49) - open/close 3282 (time 1787740611.59 total 50.05 last 59.79) - open/close 3782 (time 1787740621.60 total 60.06 last 49.96) - open/close 4321 (time 1787740631.62 total 70.08 last 53.78) - open/close 4772 (time 1787740641.63 total 80.08 last 45.07) - open/close 5249 (time 1787740651.65 total 90.11 last 47.58) - open/close 5844 (time 1787740661.68 total 100.14 last 59.35) - open/close 6423 (time 1787740671.69 total 110.14 last 57.85) - open/close 7004 (time 1787740681.69 total 120.15 last 58.05) - open/close 7605 (time 1787740691.71 total 130.16 last 60.04) - open/close 8159 (time 1787740701.73 total 140.19 last 55.24) - open/close 8710 (time 1787740711.74 total 150.20 last 55.07) - open/close 9308 (time 1787740721.74 total 160.20 last 59.80) - open/close 9731 (time 1787740731.75 total 170.21 last 42.24) total: 10000 open/close in 178.52 seconds: 56.02 ops/second llite.lustre-ffff8b2274ddb800.statahead_batch_max=0 llite.lustre-ffff8b2274ddb800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b2274ddb800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m13.061s user 0m0.351s sys 0m15.764s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8b2274ddb800.statahead_batch_max=32 llite.lustre-ffff8b2274ddb800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b2274ddb800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8b2274ddb800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m57.650s user 0m0.299s sys 0m14.307s batched statahead RPC= hit count=9975 mdc.lustre-MDT0000-mdc-ffff8b2274ddb800.batch_stats= snapshot_time: 1787740958.896371852 secs.nsecs start_time: 1787740829.405437488 secs.nsecs elapsed_time: 129.490934364 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8b2274ddb800.batch_stats=0 llite.lustre-ffff8b2274ddb800.statahead_max=2048 llite.lustre-ffff8b2274ddb800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8b2274ddb800.batch_stats=0 total: 100 open/close in 4.90 seconds: 20.40 ops/second llite.lustre-ffff8b2274ddb800.statahead_max=0 101 real 0m5.471s user 0m0.005s sys 0m0.355s 'ls -l' 100 files without statahead: 6 sec llite.lustre-ffff8b2274ddb800.statahead_max=2048 llite.lustre-ffff8b2274ddb800.statahead_max=2048 101 real 0m1.374s user 0m0.003s sys 0m0.157s 'ls -l' 100 files with statahead: 2 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10074 miss_total: 26 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 3 seconds llite.lustre-ffff8b2274ddb800.statahead_stats= statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10074 miss_total: 26 mdc.lustre-MDT0000-mdc-ffff8b2274ddb800.batch_stats= snapshot_time: 1787741361.453782709 secs.nsecs start_time: 1787740961.671726172 secs.nsecs elapsed_time: 399.782056537 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 33 33 16: 1 33 66 32: 0 0 66 64: 0 0 66 128: 1 33 100 - open/close 488 (time 1787741375.19 total 10.01 last 48.73) - open/close 1109 (time 1787741385.20 total 20.02 last 62.07) - open/close 1677 (time 1787741395.20 total 30.02 last 56.80) - open/close 2250 (time 1787741405.22 total 40.04 last 57.17) - open/close 2820 (time 1787741415.23 total 50.06 last 56.92) - open/close 3401 (time 1787741425.24 total 60.07 last 58.05) - open/close 3922 (time 1787741435.25 total 70.07 last 52.07) - open/close 4462 (time 1787741445.26 total 80.09 last 53.90) - open/close 5037 (time 1787741455.29 total 90.11 last 57.38) - open/close 5571 (time 1787741465.30 total 100.12 last 53.32) - open/close 6176 (time 1787741475.31 total 110.13 last 60.44) - open/close 6753 (time 1787741485.33 total 120.16 last 57.56) - open/close 7306 (time 1787741495.36 total 130.19 last 55.13) - open/close 7802 (time 1787741505.38 total 140.21 last 49.50) - open/close 8326 (time 1787741515.39 total 150.22 last 52.37) - open/close 8766 (time 1787741525.40 total 160.22 last 43.96) - open/close 9147 (time 1787741535.40 total 170.23 last 38.08) - open/close 9565 (time 1787741545.41 total 180.23 last 41.79) - open/close 9944 (time 1787741555.41 total 190.24 last 37.87) total: 10000 open/close in 191.45 seconds: 52.23 ops/second llite.lustre-ffff8b2274ddb800.statahead_batch_max=0 llite.lustre-ffff8b2274ddb800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b2274ddb800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m10.042s user 0m0.368s sys 0m15.770s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8b2274ddb800.statahead_batch_max=256 llite.lustre-ffff8b2274ddb800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b2274ddb800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8b2274ddb800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m57.740s user 0m0.331s sys 0m15.130s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8b2274ddb800.batch_stats= snapshot_time: 1787741707.403002545 secs.nsecs start_time: 1787741637.144203633 secs.nsecs elapsed_time: 70.258798912 secs.nsecs subreqs per batch batches %% cum %% 1: 134 51 51 2: 29 11 63 4: 10 3 67 8: 17 6 73 16: 12 4 78 32: 13 5 83 64: 7 2 86 128: 2 0 86 256: 34 13 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8b2274ddb800.statahead_batch_max=64 llite.lustre-ffff8b2274ddb800.statahead_max=128