== sanity test 123ad: Verify batching statahead works correctly ========================================================== 02:46:25 (1789713985) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff96a789d4d000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff96a789d4d000.batch_stats=0 llite.lustre-ffff96a789d4d000.statahead_max=32 llite.lustre-ffff96a789d4d000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff96a789d4d000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff96a789d4d000.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 1.07 seconds: 93.78 ops/second llite.lustre-ffff96a789d4d000.statahead_max=0 101 real 0m0.987s user 0m0.002s sys 0m0.271s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff96a789d4d000.statahead_max=32 llite.lustre-ffff96a789d4d000.statahead_max=32 101 real 0m0.424s user 0m0.005s sys 0m0.194s 'ls -l' 100 files with statahead: 0 sec statahead total: 81 statahead wrong: 2 agl total: 76 list_total: 28 fname_total: 53 hit_total: 13693 miss_total: 646 swrong=2 ewrong=2 total: 900 open/close in 3.90 seconds: 231.03 ops/second llite.lustre-ffff96a789d4d000.statahead_max=0 1001 real 0m9.142s user 0m0.038s sys 0m2.640s 'ls -l' 1000 files without statahead: 9 sec llite.lustre-ffff96a789d4d000.statahead_max=32 llite.lustre-ffff96a789d4d000.statahead_max=32 1001 real 0m3.834s user 0m0.018s sys 0m1.578s 'ls -l' 1000 files with statahead: 4 sec statahead total: 82 statahead wrong: 2 agl total: 77 list_total: 29 fname_total: 53 hit_total: 14690 miss_total: 649 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 7 seconds llite.lustre-ffff96a789d4d000.statahead_stats= statahead total: 82 statahead wrong: 2 agl total: 77 list_total: 29 fname_total: 53 hit_total: 14690 miss_total: 649 mdc.lustre-MDT0000-mdc-ffff96a789d4d000.batch_stats= snapshot_time: 1789714019.557491503 secs.nsecs start_time: 1789713987.091341359 secs.nsecs elapsed_time: 32.466150144 secs.nsecs subreqs per batch batches %% cum %% 1: 5 12 12 2: 1 2 15 4: 2 5 20 8: 1 2 23 16: 24 61 84 32: 6 15 100 mdc.lustre-MDT0001-mdc-ffff96a789d4d000.batch_stats= snapshot_time: 1789714019.557549778 secs.nsecs start_time: 1789713987.091437471 secs.nsecs elapsed_time: 32.466112307 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 2 2 8: 4 10 12 16: 11 28 41 32: 23 58 100 - open/close 2522 (time 1789714030.63 total 10.00 last 252.14) - open/close 4871 (time 1789714040.64 total 20.01 last 234.82) - open/close 6976 (time 1789714050.64 total 30.01 last 210.50) - open/close 9200 (time 1789714060.64 total 40.01 last 222.29) total: 10000 open/close in 43.50 seconds: 229.89 ops/second llite.lustre-ffff96a789d4d000.statahead_batch_max=0 llite.lustre-ffff96a789d4d000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff96a789d4d000.stats=clear mdc.lustre-MDT0001-mdc-ffff96a789d4d000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m30.065s user 0m0.241s sys 0m28.084s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff96a789d4d000.statahead_batch_max=32 llite.lustre-ffff96a789d4d000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff96a789d4d000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff96a789d4d000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff96a789d4d000.stats=clear mdc.lustre-MDT0001-mdc-ffff96a789d4d000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m38.317s user 0m0.212s sys 0m18.398s batched statahead RPC= hit count=9996 mdc.lustre-MDT0000-mdc-ffff96a789d4d000.batch_stats= snapshot_time: 1789714140.366478975 secs.nsecs start_time: 1789714097.200581217 secs.nsecs elapsed_time: 43.165897758 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 312 99 100 mdc.lustre-MDT0001-mdc-ffff96a789d4d000.batch_stats= snapshot_time: 1789714140.366551254 secs.nsecs start_time: 1789714097.200715236 secs.nsecs elapsed_time: 43.165836018 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff96a789d4d000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff96a789d4d000.batch_stats=0 llite.lustre-ffff96a789d4d000.statahead_max=2048 llite.lustre-ffff96a789d4d000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff96a789d4d000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff96a789d4d000.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 0.90 seconds: 110.52 ops/second llite.lustre-ffff96a789d4d000.statahead_max=0 101 real 0m0.638s user 0m0.002s sys 0m0.233s 'ls -l' 100 files without statahead: 0 sec llite.lustre-ffff96a789d4d000.statahead_max=2048 llite.lustre-ffff96a789d4d000.statahead_max=2048 101 real 0m0.347s user 0m0.003s sys 0m0.234s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10095 miss_total: 5 swrong=0 ewrong=0 total: 900 open/close in 2.03 seconds: 444.20 ops/second llite.lustre-ffff96a789d4d000.statahead_max=0 1001 real 0m5.847s user 0m0.024s sys 0m2.097s 'ls -l' 1000 files without statahead: 6 sec llite.lustre-ffff96a789d4d000.statahead_max=2048 llite.lustre-ffff96a789d4d000.statahead_max=2048 1001 real 0m2.939s user 0m0.015s sys 0m1.169s 'ls -l' 1000 files with statahead: 3 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11094 miss_total: 6 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 5 seconds llite.lustre-ffff96a789d4d000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11094 miss_total: 6 mdc.lustre-MDT0000-mdc-ffff96a789d4d000.batch_stats= snapshot_time: 1789714252.561758586 secs.nsecs start_time: 1789714141.347080684 secs.nsecs elapsed_time: 111.214677902 secs.nsecs subreqs per batch batches %% cum %% 1: 1 14 14 2: 2 28 42 4: 2 28 71 8: 0 0 71 16: 0 0 71 32: 0 0 71 64: 0 0 71 128: 0 0 71 256: 2 28 100 mdc.lustre-MDT0001-mdc-ffff96a789d4d000.batch_stats= snapshot_time: 1789714252.561808748 secs.nsecs start_time: 1789714141.347190459 secs.nsecs elapsed_time: 111.214618289 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 12 12 8: 3 37 50 16: 0 0 50 32: 0 0 50 64: 1 12 62 128: 1 12 75 256: 2 25 100 - open/close 4823 (time 1789714264.43 total 10.00 last 482.22) - open/close 9809 (time 1789714274.43 total 20.00 last 498.58) total: 10000 open/close in 20.42 seconds: 489.82 ops/second llite.lustre-ffff96a789d4d000.statahead_batch_max=0 llite.lustre-ffff96a789d4d000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff96a789d4d000.stats=clear mdc.lustre-MDT0001-mdc-ffff96a789d4d000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m28.795s user 0m0.176s sys 0m24.501s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff96a789d4d000.statahead_batch_max=256 llite.lustre-ffff96a789d4d000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff96a789d4d000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff96a789d4d000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff96a789d4d000.stats=clear mdc.lustre-MDT0001-mdc-ffff96a789d4d000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m23.609s user 0m0.178s sys 0m17.659s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff96a789d4d000.batch_stats= snapshot_time: 1789714333.826861070 secs.nsecs start_time: 1789714306.059266992 secs.nsecs elapsed_time: 27.767594078 secs.nsecs subreqs per batch batches %% cum %% 1: 213 58 58 2: 45 12 71 4: 30 8 79 8: 11 3 82 16: 2 0 83 32: 24 6 89 64: 2 0 90 128: 1 0 90 256: 34 9 100 mdc.lustre-MDT0001-mdc-ffff96a789d4d000.batch_stats= snapshot_time: 1789714333.826901506 secs.nsecs start_time: 1789714306.059337214 secs.nsecs elapsed_time: 27.767564292 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff96a789d4d000.statahead_batch_max=64 llite.lustre-ffff96a789d4d000.statahead_max=128