I think it's great that you support him. I don't think your mother and brother have those positions because they are old. I think it's because it's how they are and how they feel. You're the age of your brother, and you don't feel that way, right? I have a son-in-law (much younger than I am, obviously) who thinks like your mother. It's more about how people are raised and who they're exposed to through their lives.
I think it's important to watch out for labels.Dressing has nothing to do with sexual orientation/identity. So gay/bisexual/trans is not the same as cross dressing. And whether they know their "sexuality" at a young age is one thing, but having a gender identity is another - there are recent studies showing that trans people have brains like one gender even if their anatomy and their gender assignment is the opposite. It's not about hormones and sexual feelings at 4 or 5 - it's about deeply knowing who you are.
So, why are we still labeling things as "girly" or "boy toys" and then getting mad if others (like your brother) have strong ideas about what's appropriate for boys and what's not? We're contributing to the problem by keeping to those labels, you know? We are still socializing our girls to like pink things and dresses and princesses, while considering our boys "normal" if they go for trucks and weapons and super heroes.
I agree with the point made below that the "Frozen" movie has female leads - so anyone admiring their attributes is going to act and dress and sing like the women. When I was a kid, little girls didn't have these role models, so we all played Superman - no one thinks twice about us dressing as men and acting as men, do they? But when a boy does it now that some films and books have strong and capable women, ooohhh, "they're gonna be gay."
I don't think we have to cater to every child's passion-of-the-moment by buying costumes for their hero/heroines. So I'm not sure I would have bought the dress and wig. But I think it's awful that you threw them out and told your mother to explain where they went. That's kind of putting the child in the middle of your argument, of trying to prove your point. I don't think you can reason with biased people by putting them in the middle of a child's crisis now that his costume is gone.
I think you can give your nephew space to play creatively without having to prove something to his father and grandmother. There's no nice or effective way to say, "You're a bigot and I'm going to make it my mission to let this child know it." I know you're trying to support your nephew and that's great - but I think you have to respect your brother's parenting decision a little more (unreasonable as it may be) and not necessarily throw this in your mother's face.
Chances are, your nephew will not continue this for long. I think you should ease up on this, allow creative play in your school, and find a way to let him play creatively in your home without necessarily buying expensive things. Kids can make wigs out of paper bags and mop heads, so I'd encourage that rather than "I'll buy you this even though it will upset your father and grandmother." The best thing you can give this child is your time, and I think if you don't throw it in the faces of the objecting (and objectionable) relatives, you'll ensure that you'll be allowed to have that time with this child. If you make your brother mad enough and he says something in your school that puts other kids on edge or ticks off other parents, you'll have a business problem on your hand. You are a professional and need to maintain that professionalism even when your customers are your relatives.