For a point of reference, the KT266a Northbridge hits 120F/49C without any HS/fan on it at a 133FSB (actual measured data).
Yes, you should at least have a good heatsink on it. Either use an old Socket7 or Pentium heatsink or put a
Zalman heatsink on it. With a decent sized heatsink and normal air flow in a case, that will do you if you aren't OC'ing.