For the same reason, couldn't you then say W-L record shouldn't be used to rate a team, because you can have "Fake"*** Wins and Losses caused by missing players or injuries.
Margin is a generally effective way to rate a team over a large enough sample of games. There will be some one off variations, for most teams they will largely balance, but there will be the anomalies too like SU, Pitt and Bama this year who consistently performed one way in certain games.
The key thing though is that NCAA tournament is largely just selected by W and L, and the quality of those teams. It doesn't care about your margin.
*** Fake Wins is a term I stole from a Raptors board I am on. In 2021/2022 it was used by bitter fans who wanted us to tank, while we went 48-34. They would label each win as fake because somebody was usually out (especially in that Covid year where basically everybody in the league had to sit 3-4 games at various points of the year)