So I have noticed that all of my gloves wear in the lower palm:

So I went in to my local golf shop and showed him the issue. Without hesitation he said " Glove is too loose"! Never thought that was the issue. He had me put it on and showed me by pulling on the palm all the stretch in the glove. He had me go down a size and said that was perfect. Tried it today and it seemed ok. Think I will need a couple of rounds to see if that was the issue.