I think it has to do with the process of accepting your sexuality. For a straight person, there is no process. You're just "normal" and there's really nothing to worry about. Of course if you're trying to be as good a Christian as possible you have to abstain from sex, but at least you have that marriage option to have non-sinful sex.
For the gay Christian who chooses to be celibate, there is no option where he or she could enjoy non-sinful sex. That seems to add an entire different element to the psychology of the whole situation.
I could very well be wrong, and actually hope I am, because it seems like it would be really difficult for those guys.