My father loved me. He was all I had when I was little...
And I'm hoping you loved him back. I surely do understand the longing for fatherly, safe, warm nuturing. Whatever has happened over your life, you had his love back then. Too little? You needed more? Maybe, but you had that. It has, I'm guessing, given you a better foundation to living and being a caring human being then you realize.
I can't say my father ever loved me. You can Josh.
It's worth a whole lot, hang onto it as best you can. It's a set of memories that no one can take away from you. I'm tempted to suggest an exercise for you and your partner - when you're feeling really down and out - and he's in a space able to to comfort you - let him hold you and rock you and say "your father loved you", over and over. Sure it's likely to bring the watershed of tears, but maybe some little bit of peace and healing for you. Maybe something like this would help you find the answer to your question.