Gold is scarce because there's just not a lot of it in the bits of the earth we can reach.

Uranium fuel takes a lot of effort to refine, but it's still cheaper than gold; partially because it's simply more common in the crust.