Complaining about the refs shows a certain athletic immaturity. My coach always said "You don't have any control over the refs. Focus on what you DO have control over." So when I played I had control over me; when I had the ball I had control over it; when I was on defense I tried to control my man. When I coached, I worked the same angles with my kids.
When I became a ref I learned very quickly that you influence the game as much with what you call as with what you don't. So saying stuff like "Let the players decide it," is just horse hockey. That just puts the game into the hands of the teams who exploit the rules the most. So, you man up and make the right call, or find something else to do. Funny thing, though, everyone talks like we're supposed to have some magical control over everything they don't like, LOL! So anyway, today I ref pretty much the same way, I focus on what I'll call and what I won't. If the coaches don't like it, that's just the way it is. I have no control over who wins, I just make the calls or non-calls when they happen. Complaining about us isn't going to help anything, let alone help you win. If anything it detracts from your focus, and that can influence who wins.