You're right to an extent.
But Duke has already beaten Belmont, Michigan State, Michigan, and Kansas this year. Only one of those was at home. (Also lost to Ohio State). That's 4 likely NCAA tournament teams. Plus they play Washington and Temple. Last year they beat Marquette, Temple, Kansas State, Michigan State, UAB, and Butler, plus lost to St. Johns.
So in the last 2 years, I count 13-14 NCAA tournament teams they've played OOC. (Depending on how you feel about Belmont and Washington.) And very few of those at home (Belmont, Michigan State, Temple last year).
This year we have Florida, (1) who is a definite tournament team. Stanford is looking pretty good, I will throw them in. (2) State is already 5-3, with 3 pretty good losses. Don't think you can call them a tournament team.
Last year we had Michigan (3) at home, and Mich State (4) neutral. The year prior we had Cal (5) at MSG, Cornell (6) at home, Florida (7) in Tampa. The year prior, you had Florida (8) in KC, Kansas (9), and Memphis at Memphis (10), plus Cornell (11). (Did I mention I liked the schedule that year?) In 2008, we played Cornell (12). In 2007, I don't see anyone OOC. 2006, we had Florida (13) and Bucknell. (14)
So basically, Duke has played as many NCAA tournament teams OOC in the last 2 years as we have played in the last 6 years.
Edit: To be fair, I may have missed a team like Robert Morris for SU or something like that.