Part of what makes Relievers seem so "fungible" is expectations and SSS effects. There was a reliever with the Phils a few years ago that about halfway through the season had an ERA of 2.10 or so and discussion was he was having a nice, but not great season. When I looked at his game states he was having a legendary year, he had 1 game he gave up 5 ER in, in the other 27 games he pitched in he gave up nothing.
ERA is a poor stat for relievers, because one, possibly meaningless bad day can really screw up an ERA of a guy who oly pitches 70 innings or so. I am unaware of a good stat, but games without giving up a run seems like a good stat (maybe only giving half credit for games of < 1 innning, or something like that).
Great relievers are pretty consistent, but they are rare.
Very good relievers are also reasonably consistent. Ryan Madison was a solid, consistent reliever for us for years.
Most relievers are guys that have 2 pitches, 1 of which may be above average (often FB). The problem with that is that if you only have 1 above average pitch, well some guys are adept at hitting just 1 pitch really well. Thing about Ryan Howard in his prime, you could have put the best FB pitcher in the league out there against him and he stood a pretty good chance of hitting it 450 feet if he knew it was coming, no matter how fast. There are some hitters that you HAVE to avoid 1 pitch with, and if that pitch is your only good pitch then things look very ugly.