== sanity test 123ad: Verify batching statahead works correctly ========================================================== 13:04:53 (1782234293) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9f6b90dba800.batch_stats=0 llite.lustre-ffff9f6b90dba800.statahead_max=32 llite.lustre-ffff9f6b90dba800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9f6b90dba800.batch_stats=0 total: 100 open/close in 2.85 seconds: 35.12 ops/second llite.lustre-ffff9f6b90dba800.statahead_max=0 101 real 0m2.272s user 0m0.005s sys 0m0.364s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9f6b90dba800.statahead_max=32 llite.lustre-ffff9f6b90dba800.statahead_max=32 101 real 0m1.156s user 0m0.004s sys 0m0.190s 'ls -l' 100 files with statahead: 1 sec statahead total: 145 statahead wrong: 2 agl total: 139 list_total: 29 fname_total: 116 hit_total: 3749 miss_total: 1365 swrong=2 ewrong=2 - open/close 620 (time 1782234322.62 total 10.01 last 61.93) total: 900 open/close in 14.20 seconds: 63.37 ops/second llite.lustre-ffff9f6b90dba800.statahead_max=0 1001 real 0m26.147s user 0m0.041s sys 0m3.275s 'ls -l' 1000 files without statahead: 26 sec llite.lustre-ffff9f6b90dba800.statahead_max=32 llite.lustre-ffff9f6b90dba800.statahead_max=32 1001 real 0m12.791s user 0m0.034s sys 0m1.789s 'ls -l' 1000 files with statahead: 12 sec statahead total: 146 statahead wrong: 2 agl total: 140 list_total: 30 fname_total: 116 hit_total: 4747 miss_total: 1367 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff9f6b90dba800.statahead_stats= statahead total: 146 statahead wrong: 2 agl total: 140 list_total: 30 fname_total: 116 hit_total: 4747 miss_total: 1367 mdc.lustre-MDT0000-mdc-ffff9f6b90dba800.batch_stats= snapshot_time: 1782234404.159863555 secs.nsecs start_time: 1782234299.034142446 secs.nsecs elapsed_time: 105.125721109 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 662 (time 1782234417.28 total 10.00 last 66.20) - open/close 1364 (time 1782234427.29 total 20.00 last 70.18) - open/close 2015 (time 1782234437.29 total 30.01 last 65.08) - open/close 2739 (time 1782234447.30 total 40.02 last 72.32) - open/close 3385 (time 1782234457.30 total 50.02 last 64.59) - open/close 4010 (time 1782234467.30 total 60.02 last 62.49) - open/close 4700 (time 1782234477.31 total 70.03 last 68.97) - open/close 5373 (time 1782234487.31 total 80.03 last 67.29) - open/close 6049 (time 1782234497.32 total 90.04 last 67.55) - open/close 6755 (time 1782234507.32 total 100.04 last 70.60) - open/close 7360 (time 1782234517.32 total 110.04 last 60.48) - open/close 7952 (time 1782234527.33 total 120.04 last 59.18) - open/close 8522 (time 1782234537.33 total 130.05 last 56.97) - open/close 9268 (time 1782234547.34 total 140.06 last 74.52) - open/close 9918 (time 1782234557.35 total 150.07 last 64.95) total: 10000 open/close in 151.20 seconds: 66.14 ops/second llite.lustre-ffff9f6b90dba800.statahead_batch_max=0 llite.lustre-ffff9f6b90dba800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f6b90dba800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m13.515s user 0m0.386s sys 0m16.930s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9f6b90dba800.statahead_batch_max=32 llite.lustre-ffff9f6b90dba800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f6b90dba800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9f6b90dba800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m58.295s user 0m0.350s sys 0m16.312s batched statahead RPC= hit count=9983 mdc.lustre-MDT0000-mdc-ffff9f6b90dba800.batch_stats= snapshot_time: 1782234779.400398785 secs.nsecs start_time: 1782234645.591359805 secs.nsecs elapsed_time: 133.809038980 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9f6b90dba800.batch_stats=0 llite.lustre-ffff9f6b90dba800.statahead_max=2048 llite.lustre-ffff9f6b90dba800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9f6b90dba800.batch_stats=0 total: 100 open/close in 3.12 seconds: 32.03 ops/second llite.lustre-ffff9f6b90dba800.statahead_max=0 101 real 0m2.843s user 0m0.005s sys 0m0.395s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9f6b90dba800.statahead_max=2048 llite.lustre-ffff9f6b90dba800.statahead_max=2048 101 real 0m1.200s user 0m0.005s sys 0m0.217s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10082 miss_total: 18 swrong=0 ewrong=0 - open/close 634 (time 1782235116.81 total 10.01 last 63.32) total: 900 open/close in 13.62 seconds: 66.09 ops/second llite.lustre-ffff9f6b90dba800.statahead_max=0 1001 real 0m24.284s user 0m0.053s sys 0m3.595s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff9f6b90dba800.statahead_max=2048 llite.lustre-ffff9f6b90dba800.statahead_max=2048 1001 real 0m12.563s user 0m0.048s sys 0m1.922s 'ls -l' 1000 files with statahead: 13 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11081 miss_total: 19 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 26 seconds llite.lustre-ffff9f6b90dba800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11081 miss_total: 19 mdc.lustre-MDT0000-mdc-ffff9f6b90dba800.batch_stats= snapshot_time: 1782235202.096938150 secs.nsecs start_time: 1782234782.109776603 secs.nsecs elapsed_time: 419.987161547 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 16 16 16: 2 16 33 32: 3 25 58 64: 1 8 66 128: 0 0 66 256: 4 33 100 - open/close 772 (time 1782235214.66 total 10.00 last 77.18) - open/close 1447 (time 1782235224.66 total 20.00 last 67.50) - open/close 2018 (time 1782235234.67 total 30.01 last 57.07) - open/close 2758 (time 1782235244.69 total 40.02 last 73.88) - open/close 3234 (time 1782235254.71 total 50.05 last 47.50) - open/close 3763 (time 1782235264.73 total 60.06 last 52.80) - open/close 4381 (time 1782235274.74 total 70.08 last 61.72) - open/close 5036 (time 1782235284.76 total 80.10 last 65.36) - open/close 5660 (time 1782235294.77 total 90.11 last 62.32) - open/close 6306 (time 1782235304.78 total 100.12 last 64.54) - open/close 7003 (time 1782235314.79 total 110.12 last 69.66) - open/close 7579 (time 1782235324.80 total 120.14 last 57.53) - open/close 8238 (time 1782235334.81 total 130.15 last 65.84) - open/close 8860 (time 1782235344.82 total 140.16 last 62.14) - open/close 9514 (time 1782235354.84 total 150.18 last 65.21) total: 10000 open/close in 158.23 seconds: 63.20 ops/second llite.lustre-ffff9f6b90dba800.statahead_batch_max=0 llite.lustre-ffff9f6b90dba800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f6b90dba800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m16.767s user 0m0.390s sys 0m16.785s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9f6b90dba800.statahead_batch_max=256 llite.lustre-ffff9f6b90dba800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f6b90dba800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9f6b90dba800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m14.368s user 0m0.360s sys 0m16.970s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9f6b90dba800.batch_stats= snapshot_time: 1782235544.771543373 secs.nsecs start_time: 1782235452.774371491 secs.nsecs elapsed_time: 91.997171882 secs.nsecs subreqs per batch batches %% cum %% 1: 197 49 49 2: 68 17 66 4: 39 9 76 8: 27 6 82 16: 16 4 86 32: 16 4 90 64: 3 0 91 128: 0 0 91 256: 34 8 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9f6b90dba800.statahead_batch_max=64 llite.lustre-ffff9f6b90dba800.statahead_max=128