This is the great thing about sports discussions. People can have differing valid opinions about topics like awards and can have an interesting discussion.
I think that the Hall of Fame should include who the overwhelming consensus of knowledgeable fans think belong there. Tony Gwynn got 98% of the vote, and I suspect that close to 98% of knowledgeable fans think he belongs. I don't think anywhere near that percentage think Abreu belongs.
I think the writers do a good job of getting the right people into the Hall of Fame (the various incarnations of the Veterans Committee--not so much). They make mistakes, and some of the voters let personal feelings get in the way, especially when they worry about who gets in on the first ballot (does anyone care that Joe DiMaggio, Yogi Berra, and Harmon Killebrew didn't get in on the first ballot?). But the people they elect are almost always the ones that the vast majority of fans think should be there.
And so that means fame does matter. Otherwise, there would be an outcry that Dwight Evans is out and Gwynn and Winfield, and Ichiro (soon) are in. Even where I live, there is no outcry, just some whimpers.