If you don't plan on OC it, even a stock CPU cooler will work fine. I've got an fx-8120 in one of my esxi boxes and even with 10+ VMs running, its cool. Also, have you made sure the cpu cooler is properly seated? The ones that use the 4 plastic clips that you rotate to lock are bad about coming loose. Make sure to use a dab of thermal paste.

Which water cooler is it? You said its an all in one, is it a corsair?