Candu - (possible trigger?)
you have 2 different issues going on here. one is with the perp and the other with your brother. because your memories are spotty, it is going to be difficult to identify what was or wasn't incest. but i do have a couple of comments that may add something to your internal debate.
your brother was a fellow victim so whatever may have happened involving the abuser and the 2 of you was most likely totally at the abusers's instigation - and it was not your own choice. that would not be incest. however, if he forced both of you to act in one another's presence - whether there was contact between you or not - i can see how it would complicate everything and make it difficult to draw the line between incest or not. my guess is that it would border on it - if not actually crossing the line. and if he made you do things to each other - that is a more difficult possibility to consider.
you say that the perp was an "older firend" - which is vague. if he was a close enough friend to be treated like "one of the family" that is one situation. if he was more like a neighbor or more of an aquaintance or just a friend of yours - closer in age - rather than more of a parental or authority figure - that is another thing. in "Victims No Longer" Lew redefines incest in a way that shocked me when i first read it - and forced me to realize that i was an incest survivor. the step-dad - while not related by blood (my literal and narrow idea of the requirement for incest) was in a parental role. Lew says that any close relationship such as an honorary uncle, minister, scout leader or close friend of the family - can be considered an incest-type of abuse because of the trust and assumption of safety and protection that is expected. i don't have the book at hand so can't quote it - but i'd encourage you to look it up if you can. this was a revelation to me and hit me really hard - but in the long run - explained a lot once i faced it.
don't know if any of that is helpful - and with partial memories - it may all be theoretical, but hope it gives you another source to refer to if you haven't seen it already.
"the scariest thing about abuse of any shape or form, is, in my opinion, not the abuse itself, but that if it continues it can begin to feel commonplace and eventually acceptable."
- Alan Cumming, "Not My Father's Son"