It hurts because it means Syracuse has no universities to share the "culture" with. Our football culture is not suprisingly, relatively similar to Boston College. It's not hard to figure out why. And make no mistake, having [Insert State name] University on the front of one's sweatshirt is a big deal. When Penn State is on, the entire State rallies behind it. It also means kids can go to school there less expensively, they stay relatively closer to the school after they graduate, and they remain more connected to it. All of that impacts attendance.
Syracuse is the little brother of the NY Giants, NY Jets, and Buffalo Bills. Because we aren't State U, and there is no State U to root for, NY is not-surprisingly a "pro" State.