Non-drinkable tap water is mostly due to water not being available 24x7x365. If a water system is turned off even momentarily, the pressure drops to zero, and sewage and groundwater seep into the water pipes, making it unsafe to drink.
Countries with unreliable power grids usually don't have drinkable tap water for that reason.
It can also be because the water isn't treated (ie. it is just rainwater in the pipes). Water treatment actually isn't very expensive though - collecting the water and distributing the water are by far the biggest costs.
In the west water systems are pressurized using a large reservoir and gravity. I imagine its similar in third world countries? They have unreliable water due to the reasons they have unreliable power: poor ability to plan, not enough supply, undersized systems for current use, deferred maintenance, etc.
Just because you can drink the tap water, doesn't mean you want to drink it. Sure, if you are desperate. But most who have the means will prefer to purchase the better tasting and cleaner water from gallons, rather than drinking the treated water from the tap.
The pipes network is very long so small leaks will always happen somewhere due to accidents or wear and tear. These small leaks are enough to contaminate the downstream pipes if the water pressure drops and dirty water get in.
Countries with unreliable power grids usually don't have drinkable tap water for that reason.
It can also be because the water isn't treated (ie. it is just rainwater in the pipes). Water treatment actually isn't very expensive though - collecting the water and distributing the water are by far the biggest costs.