== sanity test 123ad: Verify batching statahead works correctly ========================================================== 05:00:18 (1780995618) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8b1911de2800.batch_stats=0 llite.lustre-ffff8b1911de2800.statahead_max=32 llite.lustre-ffff8b1911de2800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8b1911de2800.batch_stats=0 total: 100 open/close in 2.85 seconds: 35.14 ops/second llite.lustre-ffff8b1911de2800.statahead_max=0 101 real 0m2.872s user 0m0.008s sys 0m0.380s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8b1911de2800.statahead_max=32 llite.lustre-ffff8b1911de2800.statahead_max=32 101 real 0m1.190s user 0m0.004s sys 0m0.168s 'ls -l' 100 files with statahead: 1 sec statahead total: 144 statahead wrong: 2 agl total: 138 list_total: 29 fname_total: 115 hit_total: 3981 miss_total: 1429 swrong=2 ewrong=2 - open/close 603 (time 1780995647.52 total 10.00 last 60.27) total: 900 open/close in 16.64 seconds: 54.07 ops/second llite.lustre-ffff8b1911de2800.statahead_max=0 1001 real 0m26.905s user 0m0.033s sys 0m3.219s 'ls -l' 1000 files without statahead: 27 sec llite.lustre-ffff8b1911de2800.statahead_max=32 llite.lustre-ffff8b1911de2800.statahead_max=32 1001 real 0m9.466s user 0m0.040s sys 0m1.561s 'ls -l' 1000 files with statahead: 10 sec statahead total: 145 statahead wrong: 2 agl total: 139 list_total: 30 fname_total: 115 hit_total: 4978 miss_total: 1432 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 27 seconds llite.lustre-ffff8b1911de2800.statahead_stats= statahead total: 145 statahead wrong: 2 agl total: 139 list_total: 30 fname_total: 115 hit_total: 4978 miss_total: 1432 mdc.lustre-MDT0000-mdc-ffff8b1911de2800.batch_stats= snapshot_time: 1780995731.413243486 secs.nsecs start_time: 1780995623.739734137 secs.nsecs elapsed_time: 107.673509349 secs.nsecs subreqs per batch batches % cum % 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 641 (time 1780995743.80 total 10.02 last 63.98) - open/close 1255 (time 1780995753.81 total 20.03 last 61.34) - open/close 1774 (time 1780995763.81 total 30.03 last 51.87) - open/close 2361 (time 1780995773.84 total 40.06 last 58.53) - open/close 2975 (time 1780995783.84 total 50.06 last 61.40) - open/close 3572 (time 1780995793.85 total 60.07 last 59.69) - open/close 4173 (time 1780995803.85 total 70.07 last 60.07) - open/close 4725 (time 1780995813.85 total 80.07 last 55.18) - open/close 5247 (time 1780995823.87 total 90.09 last 52.13) - open/close 5704 (time 1780995833.89 total 100.11 last 45.61) - open/close 6134 (time 1780995843.90 total 110.12 last 42.94) - open/close 6735 (time 1780995853.91 total 120.13 last 60.05) - open/close 7316 (time 1780995863.91 total 130.13 last 58.08) - open/close 7947 (time 1780995873.92 total 140.15 last 63.03) - open/close 8555 (time 1780995883.93 total 150.15 last 60.79) - open/close 9232 (time 1780995893.93 total 160.15 last 67.69) - open/close 9901 (time 1780995903.93 total 170.16 last 66.86) total: 10000 open/close in 171.69 seconds: 58.24 ops/second llite.lustre-ffff8b1911de2800.statahead_batch_max=0 llite.lustre-ffff8b1911de2800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b1911de2800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m6.103s user 0m0.355s sys 0m16.643s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8b1911de2800.statahead_batch_max=32 llite.lustre-ffff8b1911de2800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b1911de2800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8b1911de2800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m59.427s user 0m0.322s sys 0m15.466s batched statahead RPC= hit count=9979 mdc.lustre-MDT0000-mdc-ffff8b1911de2800.batch_stats= snapshot_time: 1780996114.290478862 secs.nsecs start_time: 1780995981.426955774 secs.nsecs elapsed_time: 132.863523088 secs.nsecs subreqs per batch batches % cum % 1: 1 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8b1911de2800.batch_stats=0 llite.lustre-ffff8b1911de2800.statahead_max=2048 llite.lustre-ffff8b1911de2800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8b1911de2800.batch_stats=0 total: 100 open/close in 3.67 seconds: 27.28 ops/second llite.lustre-ffff8b1911de2800.statahead_max=0 101 real 0m3.200s user 0m0.005s sys 0m0.381s 'ls -l' 100 files without statahead: 4 sec llite.lustre-ffff8b1911de2800.statahead_max=2048 llite.lustre-ffff8b1911de2800.statahead_max=2048 101 real 0m0.958s user 0m0.009s sys 0m0.216s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10078 miss_total: 22 swrong=0 ewrong=0 - open/close 539 (time 1780996490.27 total 10.01 last 53.87) total: 900 open/close in 16.89 seconds: 53.28 ops/second llite.lustre-ffff8b1911de2800.statahead_max=0 1001 real 0m26.920s user 0m0.038s sys 0m3.207s 'ls -l' 1000 files without statahead: 27 sec llite.lustre-ffff8b1911de2800.statahead_max=2048 llite.lustre-ffff8b1911de2800.statahead_max=2048 1001 real 0m6.762s user 0m0.023s sys 0m1.747s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11077 miss_total: 23 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 26 seconds llite.lustre-ffff8b1911de2800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11077 miss_total: 23 mdc.lustre-MDT0000-mdc-ffff8b1911de2800.batch_stats= snapshot_time: 1780996571.213100355 secs.nsecs start_time: 1780996117.165740013 secs.nsecs elapsed_time: 454.047360342 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 1 9 45 64: 1 9 54 128: 1 9 63 256: 4 36 100 - open/close 527 (time 1780996584.51 total 10.01 last 52.66) - open/close 1051 (time 1780996594.52 total 20.02 last 52.34) - open/close 1591 (time 1780996604.53 total 30.03 last 53.96) - open/close 2166 (time 1780996614.54 total 40.04 last 57.44) - open/close 2769 (time 1780996624.54 total 50.04 last 60.29) - open/close 3362 (time 1780996634.56 total 60.05 last 59.23) - open/close 3930 (time 1780996644.56 total 70.06 last 56.75) - open/close 4520 (time 1780996654.57 total 80.07 last 58.95) - open/close 5126 (time 1780996664.58 total 90.08 last 60.54) - open/close 5749 (time 1780996674.59 total 100.09 last 62.25) - open/close 6290 (time 1780996684.60 total 110.09 last 54.05) - open/close 6796 (time 1780996694.60 total 120.10 last 50.57) - open/close 7386 (time 1780996704.63 total 130.12 last 58.86) - open/close 7940 (time 1780996714.64 total 140.13 last 55.35) - open/close 8476 (time 1780996724.64 total 150.13 last 53.60) - open/close 9089 (time 1780996734.65 total 160.14 last 61.26) - open/close 9680 (time 1780996744.65 total 170.14 last 59.08) total: 10000 open/close in 175.21 seconds: 57.07 ops/second llite.lustre-ffff8b1911de2800.statahead_batch_max=0 llite.lustre-ffff8b1911de2800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b1911de2800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m3.248s user 0m0.390s sys 0m17.269s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8b1911de2800.statahead_batch_max=256 llite.lustre-ffff8b1911de2800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b1911de2800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8b1911de2800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m1.388s user 0m0.347s sys 0m16.926s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8b1911de2800.batch_stats= snapshot_time: 1780996899.324452514 secs.nsecs start_time: 1780996823.575077026 secs.nsecs elapsed_time: 75.749375488 secs.nsecs subreqs per batch batches % cum % 1: 221 52 52 2: 76 17 69 4: 31 7 77 8: 25 5 83 16: 20 4 87 32: 18 4 92 64: 0 0 92 128: 0 0 92 256: 34 8 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8b1911de2800.statahead_batch_max=64 llite.lustre-ffff8b1911de2800.statahead_max=128