There's no real invalidation to be found in such a labeling. A fact is a fact. This person is male. "He" is the correct pronoun to use, regardless of whether or not "he" likes it or not.
I've lost track on the number of times I've seen things that really were bigotry accepted with a wink and a shrug, while those they were directed against were told to stop making a fuss.