I think part of the problem is nobody in Georgia cares about them - any non-alums in the state follow the Bulldogs. And they suffer from the same issue Boston College does - they are always buried behind about five other teams as far as popularity and coverage in Atlanta media.
Georgia Tech really screwed up when they left the SEC in the 60s, and burned so many bridges on their way out that once they realized it was a mistake, other schools (namely Miss St and Ole Miss) would not let them back in. If they were still in the SEC, they might be the “UCLA” to Georgia’s “USC” - as it is, it’s a school that’s struggled to forge an identity (in athletics) for my entire life.