The PSU needs a fan, always, so don't remove it! (otherwise, it'll still work, but as soon as you put too big a load on it, it will fry).
I wouldn't bother with a WB in the PSU. Just use a low cfm fan, and you'll be a happy camper.
About the HD: the reason that it is noisy, is because of the vibration that it generates. The reason that it's annoying to you, could be many.
The first thing that you have to realize, is that the HD will make that noise no matter what. The second, is that this vibration actually spreads throughout your case, making it a huge resonating box.
So, do you want to limit the vibration from spreading, or do you want to shut the whole thing up?
Try running the hard drive while it's NOT mounted to your case. If that's an acceptable noise level, then just use some rubber mounts.
If it's not an acceptable noise level, then get a 5 1/4 bay adapter, and we'll take it from there.