I agree in principle but not everything is useful for the public, and making code FOSS is not just a matter of uploading it somewhere - this requires documentation, regular updates, clear assignation of responsibility, etc. You'd be surprised by how strapped for resources even powerful government bureaucracies can be - committing time to foss-ing is time not spent on operational needs.

So I would argue: open source anything that can be expected to have wider reuse or transparency benefits. But don't just spam any code produced to the public.