It's not just about many groups " continually wanting to make it an issue", it's mentioned and brought up precisely because it IS an issue. Period.
If your point is we should ignore it, or not mention it, or act like it doesn't exist, that's a personal perspective, born of your own personal experience. The opposite of that is whether those in other groups have the same privilege, the same luxury, to offer their perspective, based on their own life experiences.
And let's be clear, this is not a right or privilege that was always historically available for those same groups in the past. If it matters to them to bring it up, then it shouldn't be a problem for anyone else. JMHO