The ultimate answer is ...well it depends.  Graphics can be very critical to some games.  Probably the best example I can think of is the Myst series.  In a Myst game graphics are possibly as important as the gameplay, because of the purpose of the game.  It is meant to be an immersive experience in a fantasy world.  A world that is pleasent simply just to visit and explore and look around. 
    A game like Wii Sports is on the other side of the fence.  It's purpose is to be simple and fun to play.  Graphics are not very important to a game like this. 
    The Myst series in turn also helps illustrate another point.  That graphics horsepower is not always important.  The first Myst game used static images rather than create a fully 3D world which the series evolved into doing.  In other words they sacrificed functionality, which wasn't a flaw in this game's case, for the sake of beautiful graphics.  Also take a look back at some of your Super Nintendo games like Zelda and Secret of Mana you will see that they are beautiful looking even though they are somewhat pixelated.  They certainly look a lot better than most games from the PS1/N64 era.  They are beautiful because of great art direction and not because of raw power.  All of these games took horsepower limitations into account and worked within them and still look good to this day because of that.  Many games from the PS1/N64 era don't hold up because those systems were powerful enough to do 3d but not enough to make 3D look good.
    Nintendo's idea for the Wii is not that graphics are irrelevant to a game's quality, but that its graphical horsepower is good enough to make a game look beautiful.