I can sympathize. I am married (though separated and working it out) to a guy who is an awesome dad now. He too didn't have a row model of a father, leaving his grandpa to fill in, and his mom (MIL) wasn't all that great either, had anger problems, and he always felt rejected and looked down upon. Sooo...DH is not the most gentle loving being. But he's got a good heart. We split up when DD was 7 months. He couldn't take the pressure of being a new father, working 12 hrs a day (and having a wife who wanted to do things and parent in a way he had never heard/experienced).
For him, when he comes home he seems to need 'me' time, and then he's ok. I didn't understand it at first, because if it was me, I'd be "give me my baby" but he needs wind down time, also because of the nature of his work.
As partners, we need to understand how our SO express their appreciation. For some, (like my DH) sweetness, words, etc are really hard, heck, he normally doesn't even kiss goodbye. His way to show us he loves us is by working hard, by providing, by 'taking care' of us.
HTH