Where to begin
I started where everyone starts with any research project: Google. After Google came months of poring over reviews of individual PC components to find what was the best compromise in quality, price, and raw power. You can ask questions right here on Gamespot, too. To build a PC, you need more parts than you might think:
- Motherboard
- CPU (Central Processing Unit)
- Thermal Grease (may be included with CPU)
- CPU Cooler (may be included with CPU)
- Memory
- Power Supply
- Graphic Card
- Case
- Hard Drive(s)
- Optical Drive(s)
- Floppy Drive/Card Reader (for Raid or Bios flash)
- Monitor
- Mouse
- Keyboard
- Software, including an Operating System
Of course, most people already own at least one computer and are able to salvage parts from their existing PC, like a keyboard and mouse, as I have. If you are considering a PC, the above is your shopping checklist.
Selecting components
You will have to visit Tom's Hardware (see resources, below) for the latest top components. They have excellent charts showing the performance improvement of each component at each price point that is extremely useful. For example, which video cards are most efficient at each price.
Motherboard - If you are going to splurge, make sure it is on the motherboard. The motherboard determines what you can upgrade in the future, which processor is accepted, and how many expansion cards you might need (and are allowed to use). I selected a Gigabyte motherboard based on the P35 Intel chipset. It will allow me to use newer 45nm processors that will become standard for the next 5 years or more. The price is also very good for the amount of options and backwards-compatibility I get with the board. Should I decide to venture into overclocking, the board is highly rated in that regard.
CPU - Your CPU should be selected from the motherboard manufacturer's compatibility list, available at their web site. The beefier the CPU the more Ken Jennings be your computer. I selected a Intel Core 2 Duo E6550, which is the least expensive of their newest processor line, and will allow me to run pretty much everything conceivable at reasonable quality. I can always upgrade when prices come down, if the need is warranted; CPUs are easy to swap. The retail package includes a cooler, which mounts on top of the processor, and preloaded thermal grease or a pad. The included thermal grease or pad is generally inferior to aftermarket grease such as Arctic Silver, and aftermarket grease is cheap at five or ten bucks, so I picked some up.
Memory - Should be selected from the motherboard manufacturer's compatibility list, just like the CPU. I selected two gigs of Crucial Ballistix memory based on positive customer feedback at Newegg and compatibility, but mostly because of a $30 rebate that put it below $80. The timings, an indication of speed, were also good at 4-4-4-12 and a CAS of 4. In memory, lower numbers are generally better, but may require more technical skill in setting up the PC, such as adjusting the voltage in the BIOS (more on this later, if it becomes necessary).
Power supply - They vary in price more than any other component, and technology improves little from year to year, unlike processors and memory. Wait for a good deal on a positively-reviewed power supply and buy it at any time, even if your PC will not be built for awhile. Power supplies will not go bad with disuse, and will not become obsolete with time if it is high quality. Just remember, the more stuff you plan on running, the beefier a power supply you will need. I am only going to run a single graphics card, so the 500W Thermaltake Purepower, on special with free shipping and a $20 rebate, will do me fine.
Graphics card - The easiest part of the buying process; I went to Tom's Hardware, clicked their current chart for the best power/price ratio in my price target ($150-$160), and selected a Sapphire Radeon X1950PRO. The card is certainly not top-of-the-line, but will get me through a couple years of gaming and Photoshop.
Case - Some are easier to work in than others, but unless you need a small form-factor PC any ATX case will generally do fine, you pay extra for bells & whistles. I picked up a really nice Ultra Wizard ATX case for free after rebate. You can't beat free. 
Hard drive - Easy: buy as much as you can afford. Storage is a mature technology, and it is getting constantly cheaper to store more information on smaller disks. Even if you buy a small hard drive today, odds are you can pick one up even cheaper in the future. A $55, 200GB Western Digital is more than enough for my purposes, as I will also be salvaging a 160GB PATA (the interface type) hard drive from my old PC, great for storing music and *ahem* videos. The newer SATA interface on the Western Digital will be faster than the PATA drive.
Optical drives - Just as easy to select as hard drives. Any DVD burner rated highly on Newegg will work well for most users. I selected a Lite-On that is on special with free shipping.
The remaining components were salvaged from my old PC with the exception of the monitor, which will be determined by your needs. I picked up a 22" Acer from Woot a number of months ago that I will be using. I will be using Windows XP. Vista is simply not mature enough to warrant purchase, especially in the three-digit price range.
The final list of my components:
Intel Core 2 Duo E6550 Conroe 2.33GHz ($179)
GIGABYTE GA-P35-DS3R ($130)
SAPPHIRE Radeon X1950PRO 256MB DDR3 ($137)
Crucial Ballistix 2GB 4-4-4-12 ($110, $80 after rebate)
LITE-ON 20X DVD±R (SATA) ($29)
Thermaltake Purepower 500W ($60, $30 after rebates)
Western Digital Caviar SE 200GB 7200 RPM SATA 3.0Gb/s ($55)
Ultra Wizard ATX Case ($40, free after rebate)
Total Cost: $640 plus shipping 
Placing the order
Multiple special discounts forced me to purchase my items from Newegg in separate orders (you do what you have to for the discounts!). I already have my case, splayed open on a table at home, awaiting the components that I will place into its currently empty shell. Newegg is fast, and the components arrived the day after order placement. Impressive!
I began installing the components last night, and will take you along with me to learn how to build and troubleshoot a working PC.
Resources
PCStats - Beginner's Guide to PC Building
Tom's Hardware - Hardware reviews and recommendations
Anandtech - Hardware reviews and recommendations
ExtremeTech - Hardware reviews and recommendations
Bootdisk.com - Excellent resource for BIOS flashing and other basics
Newegg - Online PC Supply Store
Slickdeals - Collection of current bargains (the best)
Ben's Bargains - Collection of current bargains
Freeafterrebate - Collection of free stuff (sort of)