This is undoubtedly true for some fans, and may be true for most - but not for all. For some of us, the performances are what is engaging - can so-and-so achieve a new personal best, etc. There's an aesthetic to be appreciated, apart from the "we won" aspect.
I confess, I don't entirely get the "rivalry" aspect, as it translates to "I hate the Mets!" (or whoever). I don't get a different satisfaction from beating the Mets (or the Braves, etc.), as opposed to, say, Cleveland. I want the Phils to win - but that's because I want them to play well, to achieve, not because they're not "the bad guys." But I know that others differ, and that's fine; I do sometimes have to wonder if some people know what the word "hate" actually means...
So I don't need a superpower rivalry to enjoy the Olympics. If the Russian pairs skaters are the best in the world, I want them to do well, to excel, because I want to see that. I don't want a foreign skier to fall so an American who's not as good can win. I have no desire to see Caster Semenya banned because her body's hormone level isn't "typical," so somebody who's not as fast can "win."
At the end of the day, the Olympics - sport in general - isn't just (or primarily) about winning. It's about striving, it's about becoming better, at whatever you do. Vince Lombardi was wrong.
Americans - by and large - don't get it, IMHO.