I find it helpful to consider the older sense of the word itself, which when examined does not contain malice, only absolute refusal; it was originally used as a less passive form of "abhor", closer to "shun" or "to disallow w/ extreme prejudice", than what its commonly understood meaning is today. Re-read the KJV Bible, or some Elizabethan works to get a better sense of this. Not trying to be "mr. language snob", but sometimes it helps to explore where our words come from, it helps me avoid confusing myself or others.

Give sorrow words: the grief that does not speak
Whispers the o'er-fraught heart and bids it break.

-William Shakespeare, Macbeth, Act IV, Sc. III