What would happen to the capacitance of a capacitor on replacing the dielectric inside with a conductor?

Application of what i know tells me that it would be infinite. But then if it did, why use dielectrics in capacitors in the first place..?