My favorite is still Phantom City Records located at 304 4th Avenue East. It's downtown and shares a space with Dumpster Values. They carry metal, noise, punk, and indie rock, both new and used. They probably have more vinyl than compact discs.
Update (Feb 2012): Phantom City has changed hands again, is has been on steady decline ever since then. They no longer accept plastic. Their selection has really shrunk. Most of the K Records and Kill Rock Stars releases are gone, never to be restocked. The metal section is slowly withering away. Instead of being excited to come here, I approach the store with trepidation. It's like watching my best friend succumb to the ravages of drug addiction, and there's nothing I can do about it until it's too late.
Update (Aug 2013): Phantom City relaunched a few months ago. No more CDs. Their vinyl selection grew, and now they are getting some pretty decent material, both new and used. I hope this trend will continue.
Update (July 2014): Phantom City is dead.
Update (August 2014): Funk Fuzz has taken over the Phantom City location. I went by today and checked them out. The store is primarily vinyl, but does have some CDs. The selection is electic: there is still some hardcore / punk / indie rock, but they also carry mainstream (70's & 80's) vinyl and quite a bit of of odd modern psych. There is even a small Finnish section where I was able to pick up Ajaton Vie by Aan. They are cash only.
Located about one block away, at 301 5th Avenue SE is Olympia's oldest surviving record store. Rainy Day Records used to be on the other side of the bridge before they moved downtown. They carry a large selection of vinyl and CDs from hip hop to indie rock to garage rock, both new and used. They also carry a small selection of turntables and skateboards, as well as a dwindling supply of Converse shoes that were made in the USA. Rainy Day also offers DVD/Video rentals and they specialize in independent and foreign films.
Over in Lacey is Boomerang. They're located in the strip mall adjacent to Fred Meyer. They mostly offer used CDs and some used vinyl, as well as a growing supply of used DVDs. They have a small metal selection.
Update (March 2014): Boomerang has moved across the street to 4129 Pacific Avenue. They are in the same building that Olympic Cards & Comics used to be in. They have a little more room, but the prices seem to have risen. There is even a selection of material that they have listed online, for very inflated prices.
Where to See Bands:
Olympia still has plenty of house shows. There might be some flyers posted downtown. The China Clipper and The Voyeur, both on 4th host shows irregularly. Recently, there have been quite a few shows at The Northern. It's an art gallery on 4th near Le Voyeur.
Update (March 2014): The Northern moved some time ago to 414 1/2 Legion Way .
I've been to a few shows at The Hall Of The Woods. Wolves In The Throne Room have played many shows there. I've seen Bloodhag perform at both the downtown Olympia library and the Tumwater library.
Located downtown on 5th Avenue is the Capitol Theater. They show a lot of foreign and independent films. They host a film festival every year. Bands play here infrequently, so check their website. When the Melvins swing through town, this is where they play.
Sadly, most bands still skip the Pacific Northwest when they tour. There's not too many places to play even in Seattle or Portland, so it's probably not worth the time and money. Some bands, like The Melvins and Sunn O))), play here on almost every tour.