== sanity test 123ad: Verify batching statahead works correctly ========================================================== 06:19:13 (1788689953) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff954360edd000.batch_stats=0 llite.lustre-ffff954360edd000.statahead_max=32 llite.lustre-ffff954360edd000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff954360edd000.batch_stats=0 total: 100 open/close in 3.09 seconds: 32.33 ops/second llite.lustre-ffff954360edd000.statahead_max=0 101 real 0m1.922s user 0m0.008s sys 0m0.291s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff954360edd000.statahead_max=32 llite.lustre-ffff954360edd000.statahead_max=32 101 real 0m1.122s user 0m0.008s sys 0m0.133s 'ls -l' 100 files with statahead: 1 sec statahead total: 121 statahead wrong: 2 agl total: 112 list_total: 30 fname_total: 77 hit_total: 3615 miss_total: 1408 swrong=2 ewrong=2 - open/close 822 (time 1788689981.28 total 10.01 last 82.12) total: 900 open/close in 11.16 seconds: 80.62 ops/second llite.lustre-ffff954360edd000.statahead_max=0 1001 real 0m26.901s user 0m0.034s sys 0m2.865s 'ls -l' 1000 files without statahead: 27 sec llite.lustre-ffff954360edd000.statahead_max=32 llite.lustre-ffff954360edd000.statahead_max=32 1001 real 0m10.620s user 0m0.028s sys 0m1.193s 'ls -l' 1000 files with statahead: 10 sec statahead total: 122 statahead wrong: 2 agl total: 113 list_total: 31 fname_total: 77 hit_total: 4612 miss_total: 1411 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff954360edd000.statahead_stats= statahead total: 122 statahead wrong: 2 agl total: 113 list_total: 31 fname_total: 77 hit_total: 4612 miss_total: 1411 mdc.lustre-MDT0000-mdc-ffff954360edd000.batch_stats= snapshot_time: 1788690052.417200145 secs.nsecs start_time: 1788689958.700389296 secs.nsecs elapsed_time: 93.716810849 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 547 (time 1788690065.31 total 10.02 last 54.60) - open/close 1179 (time 1788690075.32 total 20.03 last 63.15) - open/close 2142 (time 1788690085.32 total 30.03 last 96.27) - open/close 3050 (time 1788690095.33 total 40.04 last 90.71) - open/close 3820 (time 1788690105.34 total 50.05 last 76.93) - open/close 4515 (time 1788690115.34 total 60.05 last 69.47) - open/close 5362 (time 1788690125.35 total 70.06 last 84.67) - open/close 6274 (time 1788690135.35 total 80.06 last 91.15) - open/close 7150 (time 1788690145.35 total 90.06 last 87.58) - open/close 7893 (time 1788690155.36 total 100.07 last 74.24) - open/close 8436 (time 1788690165.37 total 110.08 last 54.24) - open/close 9055 (time 1788690175.37 total 120.08 last 61.90) - open/close 9851 (time 1788690185.37 total 130.08 last 79.60) total: 10000 open/close in 131.72 seconds: 75.92 ops/second llite.lustre-ffff954360edd000.statahead_batch_max=0 llite.lustre-ffff954360edd000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff954360edd000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m2.762s user 0m0.307s sys 0m13.321s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff954360edd000.statahead_batch_max=32 llite.lustre-ffff954360edd000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff954360edd000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff954360edd000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m44.515s user 0m0.262s sys 0m12.599s batched statahead RPC= hit count=9966 mdc.lustre-MDT0000-mdc-ffff954360edd000.batch_stats= snapshot_time: 1788690373.881347739 secs.nsecs start_time: 1788690258.332691539 secs.nsecs elapsed_time: 115.548656200 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff954360edd000.batch_stats=0 llite.lustre-ffff954360edd000.statahead_max=2048 llite.lustre-ffff954360edd000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff954360edd000.batch_stats=0 total: 100 open/close in 2.87 seconds: 34.86 ops/second llite.lustre-ffff954360edd000.statahead_max=0 101 real 0m2.586s user 0m0.004s sys 0m0.304s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff954360edd000.statahead_max=2048 llite.lustre-ffff954360edd000.statahead_max=2048 101 real 0m0.979s user 0m0.003s sys 0m0.152s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10065 miss_total: 35 swrong=0 ewrong=0 - open/close 870 (time 1788690668.78 total 10.00 last 86.97) total: 900 open/close in 10.31 seconds: 87.28 ops/second llite.lustre-ffff954360edd000.statahead_max=0 1001 real 0m18.856s user 0m0.029s sys 0m2.641s 'ls -l' 1000 files without statahead: 18 sec llite.lustre-ffff954360edd000.statahead_max=2048 llite.lustre-ffff954360edd000.statahead_max=2048 1001 real 0m5.293s user 0m0.026s sys 0m1.220s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11064 miss_total: 36 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 17 seconds llite.lustre-ffff954360edd000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11064 miss_total: 36 mdc.lustre-MDT0000-mdc-ffff954360edd000.batch_stats= snapshot_time: 1788690722.501946243 secs.nsecs start_time: 1788690376.320898588 secs.nsecs elapsed_time: 346.181047655 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 758 (time 1788690735.21 total 10.00 last 75.80) - open/close 1559 (time 1788690745.23 total 20.02 last 79.95) - open/close 2268 (time 1788690755.23 total 30.02 last 70.88) - open/close 2993 (time 1788690765.24 total 40.03 last 72.42) - open/close 3920 (time 1788690775.24 total 50.04 last 92.68) - open/close 4781 (time 1788690785.26 total 60.05 last 86.00) - open/close 5590 (time 1788690795.26 total 70.05 last 80.87) - open/close 6424 (time 1788690805.29 total 80.08 last 83.19) - open/close 7214 (time 1788690815.30 total 90.09 last 78.91) - open/close 8017 (time 1788690825.31 total 100.10 last 80.20) - open/close 8785 (time 1788690835.33 total 110.12 last 76.65) - open/close 9576 (time 1788690845.33 total 120.12 last 79.08) total: 10000 open/close in 125.31 seconds: 79.80 ops/second llite.lustre-ffff954360edd000.statahead_batch_max=0 llite.lustre-ffff954360edd000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff954360edd000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m55.250s user 0m0.314s sys 0m13.356s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff954360edd000.statahead_batch_max=256 llite.lustre-ffff954360edd000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff954360edd000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff954360edd000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m50.505s user 0m0.254s sys 0m13.023s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff954360edd000.batch_stats= snapshot_time: 1788690974.949307991 secs.nsecs start_time: 1788690914.045576298 secs.nsecs elapsed_time: 60.903731693 secs.nsecs subreqs per batch batches %% cum %% 1: 23 20 20 2: 9 8 29 4: 2 1 30 8: 8 7 38 16: 11 10 48 32: 15 13 61 64: 4 3 65 128: 3 2 68 256: 35 31 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff954360edd000.statahead_batch_max=64 llite.lustre-ffff954360edd000.statahead_max=128