The NCAA works three-man, and I only do two-man games, usually 3 nights/wk. I'm not "so out of shape," I'm 5-11 and 175. Most of my colleagues are of a similar body type. I used to work college games back in my late 30's - early 40's. Then I took a job teaching & coaching, and there was no time to ref, so I took those 15 years off. Then when I retired from teaching, I told the assn I was not interested in the politics of climbing back up the ladder, just wanted to have some fun reffing HS games, and make a little for beer and green fees. So I just do 2-man.
It's a dichotomy, frankly. The three-man system was put in place because the off-ball activity was practically out of control, especially in transitional play. The down side of it is that it extended the careers of the dinosaurs for they don't have to run nearly as much. But let's be honest here, they're not all fat and out of shape. Very few are anymore. You're thinking of Burr, Higgins, Clougherty, et al, from the old BE days. Today the assn's have moved toward getting those guys out of the equation. But here's something else to consider. I learned as a coach that I don't care much what the ref looks like, or how he runs or if he wears glasses or has a beard, if he's getting the calls right. So I'd rather have the three-man with "rounder" refs if they're getting the calls right. At the end of the game, that's what matters.