To be fair to the old man, I was going to see if times that Brady didn't play due to injury or suspension impacted the record since expectations should be lowered due to using a backup. However, Googles AI says that the Patriots went 20-16 in the games where Brady was not the starting quarterback during the years he was with the team.
[Edit: I hit POST, then realized that this would include the end of Bledsoe's time when Brady was the backup. Looks like a more accurate count is 15 games injured in 2008 (they went 10-5) and 5 games deflategated in 2020 (they went 2-3).]