Treadmill_Guy you seem to be talking about acting out - or thinking about acting out - when you say this:
I don't find men physically attractive. Women always draw my attention. But there is something about a man's penis that draws my interest and fantasies.
Part of the reason a survivor may feel this way has to do with control, as someone else has already said. As a boy you learned from abuse that you were helpless and powerless; the abuse involved loss of control. So in these fantasies you are creating scenarios were your control is restored.
You might ask why you would create a scenario in which you are still performing sexual acts reminiscent of the abuse. Again the answer lies in your youth. Abuse can leave a boy with no boundaries and feeling he is good only for what the abuser wants. Do you remember feeling like an empty shell with no "me" inside anymore?
You fantasize a sexual situation in which you are back in control, but the situation is one that reflects the negative feelings you learned as an abused boy.