Persona 4 is a unique experience, and certainly worth playing.
My first impressions of the game (I haven't played the previous ones) was that it was good, but not particularly accessible for western audiences. 60 hours later, I had the same feeling.
Persona 4 has in it everything that a good RPG needs - a good story and good gameplay. It throws in excellent characters and a very unique setting and atmosphere to make it a fresh experience. The combat is good, but not great, and I never got over how weird the enemies are. The whole experience was quite strange, and most of the times I didn't enjoy the dungeon crawling bits as much as the rest of the game.
Besides that, there were a few other annoyances. Most notably the audio. The voice acting is fine, but man, the music. Music is important, Games like the windwaker show you just how much it transforms a game. So I love VGM, and this made Persona's 4 japanese pop music extremely infuriating.
Overall, the best Persona 4 has to offer is really good, but I can understand why some people wouldn't like it. If you do like it, you'll love it, as it has a massive play length.