In the real world, people do in fact get a guilty verdict only to discover the individual wasn't guilty.

The scenario described is rare, but what you are advocating for is to overfit at the cost of a few innocents. Which is great, until it happens to you or someone you care for.

The world is constantly changing. Law and justice aren't rigid. You never know when you find yourself on the other side with no way of realizing.