== sanity test 123ad: Verify batching statahead works correctly ========================================================== 17:51:36 (1788817896) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff92825d5de800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff92825d5de800.batch_stats=0 llite.lustre-ffff92825d5de800.statahead_max=32 llite.lustre-ffff92825d5de800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff92825d5de800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff92825d5de800.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 3.69 seconds: 27.11 ops/second llite.lustre-ffff92825d5de800.statahead_max=0 101 real 0m3.125s user 0m0.003s sys 0m0.296s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff92825d5de800.statahead_max=32 llite.lustre-ffff92825d5de800.statahead_max=32 101 real 0m1.202s user 0m0.001s sys 0m0.188s 'ls -l' 100 files with statahead: 2 sec statahead total: 135 statahead wrong: 2 agl total: 129 list_total: 29 fname_total: 96 hit_total: 4115 miss_total: 731 swrong=2 ewrong=2 - open/close 814 (time 1788817930.42 total 10.01 last 81.29) total: 900 open/close in 11.35 seconds: 79.31 ops/second llite.lustre-ffff92825d5de800.statahead_max=0 1001 real 0m30.582s user 0m0.034s sys 0m2.853s 'ls -l' 1000 files without statahead: 30 sec llite.lustre-ffff92825d5de800.statahead_max=32 llite.lustre-ffff92825d5de800.statahead_max=32 1001 real 0m8.882s user 0m0.023s sys 0m1.291s 'ls -l' 1000 files with statahead: 9 sec statahead total: 136 statahead wrong: 2 agl total: 130 list_total: 30 fname_total: 96 hit_total: 5111 miss_total: 735 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff92825d5de800.statahead_stats= statahead total: 136 statahead wrong: 2 agl total: 130 list_total: 30 fname_total: 96 hit_total: 5111 miss_total: 735 mdc.lustre-MDT0000-mdc-ffff92825d5de800.batch_stats= snapshot_time: 1788818005.595540313 secs.nsecs start_time: 1788817903.136838238 secs.nsecs elapsed_time: 102.458702075 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 2 5 7 8: 4 10 17 16: 18 45 62 32: 15 37 100 mdc.lustre-MDT0001-mdc-ffff92825d5de800.batch_stats= snapshot_time: 1788818005.595685896 secs.nsecs start_time: 1788817903.136912478 secs.nsecs elapsed_time: 102.458773418 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 1 2 5 4: 2 5 10 8: 3 7 17 16: 17 42 60 32: 16 40 100 - open/close 915 (time 1788818017.90 total 10.00 last 91.46) - open/close 1792 (time 1788818027.91 total 20.01 last 87.68) - open/close 2683 (time 1788818037.91 total 30.01 last 89.10) - open/close 3544 (time 1788818047.93 total 40.03 last 85.94) - open/close 4494 (time 1788818057.95 total 50.05 last 94.77) - open/close 5443 (time 1788818067.96 total 60.06 last 94.76) - open/close 6394 (time 1788818077.98 total 70.08 last 94.97) - open/close 7458 (time 1788818087.98 total 80.08 last 106.38) - open/close 8493 (time 1788818098.00 total 90.10 last 103.33) - open/close 9350 (time 1788818108.00 total 100.10 last 85.67) total: 10000 open/close in 107.78 seconds: 92.78 ops/second llite.lustre-ffff92825d5de800.statahead_batch_max=0 llite.lustre-ffff92825d5de800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92825d5de800.stats=clear mdc.lustre-MDT0001-mdc-ffff92825d5de800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m3.554s user 0m0.284s sys 0m14.042s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff92825d5de800.statahead_batch_max=32 llite.lustre-ffff92825d5de800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92825d5de800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff92825d5de800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff92825d5de800.stats=clear mdc.lustre-MDT0001-mdc-ffff92825d5de800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m41.498s user 0m0.288s sys 0m13.228s batched statahead RPC= hit count=9979 mdc.lustre-MDT0000-mdc-ffff92825d5de800.batch_stats= snapshot_time: 1788818297.614490705 secs.nsecs start_time: 1788818187.037929115 secs.nsecs elapsed_time: 110.576561590 secs.nsecs subreqs per batch batches %% cum %% 1: 1 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff92825d5de800.batch_stats= snapshot_time: 1788818297.614558703 secs.nsecs start_time: 1788818187.038060585 secs.nsecs elapsed_time: 110.576498118 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff92825d5de800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff92825d5de800.batch_stats=0 llite.lustre-ffff92825d5de800.statahead_max=2048 llite.lustre-ffff92825d5de800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff92825d5de800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff92825d5de800.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 2.06 seconds: 48.56 ops/second llite.lustre-ffff92825d5de800.statahead_max=0 101 real 0m2.191s user 0m0.006s sys 0m0.302s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff92825d5de800.statahead_max=2048 llite.lustre-ffff92825d5de800.statahead_max=2048 101 real 0m0.925s user 0m0.009s sys 0m0.163s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10078 miss_total: 22 swrong=0 ewrong=0 - open/close 743 (time 1788818577.18 total 10.01 last 74.25) total: 900 open/close in 12.99 seconds: 69.31 ops/second llite.lustre-ffff92825d5de800.statahead_max=0 1001 real 0m23.654s user 0m0.037s sys 0m2.928s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff92825d5de800.statahead_max=2048 llite.lustre-ffff92825d5de800.statahead_max=2048 1001 real 0m11.282s user 0m0.022s sys 0m1.337s 'ls -l' 1000 files with statahead: 12 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11077 miss_total: 23 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff92825d5de800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11077 miss_total: 23 mdc.lustre-MDT0000-mdc-ffff92825d5de800.batch_stats= snapshot_time: 1788818652.924749746 secs.nsecs start_time: 1788818300.510386231 secs.nsecs elapsed_time: 352.414363515 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 11 11 4: 2 22 33 8: 2 22 55 16: 0 0 55 32: 2 22 77 64: 0 0 77 128: 0 0 77 256: 2 22 100 mdc.lustre-MDT0001-mdc-ffff92825d5de800.batch_stats= snapshot_time: 1788818652.924817631 secs.nsecs start_time: 1788818300.511086010 secs.nsecs elapsed_time: 352.413731621 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 11 11 4: 1 11 22 8: 2 22 44 16: 2 22 66 32: 1 11 77 64: 0 0 77 128: 0 0 77 256: 2 22 100 - open/close 906 (time 1788818665.44 total 10.01 last 90.55) - open/close 1863 (time 1788818675.44 total 20.01 last 95.67) - open/close 2866 (time 1788818685.44 total 30.01 last 100.26) - open/close 3840 (time 1788818695.45 total 40.02 last 97.35) - open/close 4742 (time 1788818705.45 total 50.02 last 90.14) - open/close 5725 (time 1788818715.47 total 60.03 last 98.19) - open/close 6547 (time 1788818725.48 total 70.05 last 82.11) - open/close 7468 (time 1788818735.48 total 80.05 last 92.08) - open/close 8318 (time 1788818745.48 total 90.05 last 84.98) - open/close 9041 (time 1788818755.49 total 100.05 last 72.27) - open/close 9855 (time 1788818765.49 total 110.06 last 81.34) total: 10000 open/close in 111.80 seconds: 89.44 ops/second llite.lustre-ffff92825d5de800.statahead_batch_max=0 llite.lustre-ffff92825d5de800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92825d5de800.stats=clear mdc.lustre-MDT0001-mdc-ffff92825d5de800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m59.372s user 0m0.289s sys 0m13.915s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff92825d5de800.statahead_batch_max=256 llite.lustre-ffff92825d5de800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92825d5de800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff92825d5de800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff92825d5de800.stats=clear mdc.lustre-MDT0001-mdc-ffff92825d5de800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m52.925s user 0m0.298s sys 0m13.766s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff92825d5de800.batch_stats= snapshot_time: 1788818898.766926852 secs.nsecs start_time: 1788818834.467197286 secs.nsecs elapsed_time: 64.299729566 secs.nsecs subreqs per batch batches %% cum %% 1: 128 43 43 2: 45 15 59 4: 34 11 70 8: 19 6 77 16: 10 3 80 32: 13 4 84 64: 8 2 87 128: 2 0 88 256: 34 11 100 mdc.lustre-MDT0001-mdc-ffff92825d5de800.batch_stats= snapshot_time: 1788818898.766988716 secs.nsecs start_time: 1788818834.467468181 secs.nsecs elapsed_time: 64.299520535 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff92825d5de800.statahead_batch_max=64 llite.lustre-ffff92825d5de800.statahead_max=128