Ok, went out to Lowes and found a nice Delta 10" drill press and picked it up for $95. Seems pretty rigid and the bearings are nice and smooth (a crapload better than my dad's massive 500lb drill press) It also seems that the only tapered fitting goes directly into the chuck, so that should be easy to heat up and hammer on. I doubt it would come off if I used a little loctite too (maybe JB weld?) The chuck seems pretty solid and very well made. It looks like quite a bit of machine time went into making it. Very well built little thing. Motor is smooth and has no vibration. Going to set up this little thing on it today, it is a little dial thing that measures deflection in a rotating object up to .0001", that should see if it is running true.
I'm definitely going to have to take the band saw to the post because it is way too tall for milling. That should remove a lot of the rigidity problems right there. I'm also going to weld it to the head unit because There is no way the two little allen screws can hold it in there well.
I'll take some pics of the thing tonight. No turning back now
