In my experience it really depends on how close the holes are.
Pilot holes also
REALLY help. If you dont centre punch/drill pilot holes, and your drill holes are close enough together the bit sometimes wanders right into the other hole, even before you've gotten started on the hole. But pilot holes are only a start, once you go deep enough with the final size, there is sometimes a tendency for the drill bit to try to "grab" or pull into the adjacent hole. (I've had that on a couple occasions, unpleasent noises, etc.)
Lining the drill bit up without touching, and then giving it a "quick" start also seems to help, don't "dilly dally" the start of a hole, just get it in there - I've ruined many a block that way (well maybe 1 at the most

)
Practice practice practice - did I mention practice? I was drilling a #rotor style block yesterday (? maybe the day before) and after nearly completing it I decided I should start again (alright alright, I drilled straight through stupid me...). Moral of the story is, measure twice cut once - no actually it was supposed to be that after that many drill holes (I did some full depth and some 1/2 depth pilot holes) so after both blocks like 200 holes or something - you get to know what's going to happen a bit - you get used to the material/bit and drill speeds.
Maybe BladeRunner will recomment on the situation now he has the milling experience vs the his old approach. He seemed to have the drilling / spacing /channeling thing down to a fine art.
[PS: Wow, I walk upstairs and I have a soothing LCD and a
functional KB how amazing

]