Maybe I can shed some light...
0 ohm = full load = closed circuit = short = superconductor
infinite ohm = no load = open circuit
Batteries have the potential to provide 1.5V (AA). The potential is there, wether it is used or not. If it is used, it can be measured. If it is not used, then it is just potential, but is still measured in volts.
A battery can only supply so much power. If it is shorted, then the voltage would drop some, as the short tries to draw everything out of it. The voltage would drop to where the amperage is high, but to where the maximum power is outputed. (outputed?)
Batteries also have an internal resistance, but I guess that doesn't mean anything here.