Places where I've found obscure goodies and been pleased with the service:
www.aquariusrecords.org (I'm assuming everyone knows this place, but if not, you owe it to yourself to check out their site- there are a huge number of sound samples of music that's impossible to check out anywhere else online, for a start)
www.normanrecords.com UK retailer, good avant/experimental selection
www.glitterhouse.de Found a couple of things here that couldn't be found elsewhere- very fast shipping from Germany to UK.
http://rermegacorp.com/ You all know this one hopefully...
www.freakemporium.com All things psychedelic, as the name suggests, though a fair selection of punk, reggae, film soundtracks etc. too. They send out occasional print catalogues which are things of beauty, if you like poring over pictures and descriptions of long-forgotten early 70s obscurities, which I'm sure you all do.
www.musicstack.com Huge meta-site that collects data from hundreds of record dealers and allows you to compare prices and order- prices go suprisingly low at times.
More as I think of them.