I'm somewhat familiar with it, but all I've heard you say is that it's a fantasy you have. I have all kinds of fantasies that don't get in my way.
The core issue is whether it is an obsession or fixation that prevents you from having a satisfying relationship with any other man, regardless of his height.
In macrophilia there is a lot of power imbalance, control, and size difference elements, phallus, nipple, and muscle worship, too. In many ways the focus on really masculine elements doesn't seem much different to the muscle morphing sub-culture.
Do you think that unless you can find a giant (exactly how big would they need to be) you won't be able to have a relationship with a man? It would be worth understanding what it is about the giant fantasy that rings your bell. Is that something you've been getting into with your therapist?