64 Bit Performance?

Specs:

Gigabyte EP35-DS3L
Intel Core 2 Duo E7200 @ 2.85ghz
2x2gb G.Skill DDR2 800
MSI R4850 @ 700/1100

I was wondering if I would see an improvement in overall performance by switching from Windows XP 32 bit to Windows XP 64 bit or Vista 64 bit, and if so, should I go with XP 64 or Vista 64? One of the things I should mention is that Windows XP 32 is only recognizing my RAM as 3.5gb, not the full 4 gb. Just wanted to ask to see if it would be worth it for gaming and other applications, I have a copy of XP and Vista 64 just haven't used them.

Thanks.