Don't worry you're doing fine on balance it seems to me. You've been with a woman for four years out of the last six from what I make out and you'll be missing that probably. But it'll sort itself out in time. That you can get sex shows at least you're physically attractive to women. Not every man can feel that.
I was 23 also before I stopped being a virgin, they had me in mind when they came up with the phrase crazy mixed up kid. I think I slept with another four or five women after that myself, the last one is still with me. We got married forty years ago last Christmas Eve.
Good luck Kirby it's never easy but we get there.