I've worked in a support team for a telecom billing system and I was tasked with interacting with our development team to investigate bugs in production (and eventually moved to the development team). These systems are created just like any other commercial system, without any formal proofs and minimum requirement docs. To make things worse, they have to be flexible enough to support all and any billing plans that the business might come up with, so there is a lot of moving parts.
As other people have said here, nobody wants to touch it. Developers would often limit themselves to fix just a small portion of the code even though they thought the overall system could be improved in many ways, for fear of breaking something, causing a few million dollars of damage and getting fired. There was no assurance that any part of the systems should work like this or that.. only some vague expectations.
You're right, that would be a systems that should be built from scratch with that kind of concern but unfortunately it's not.
It is a nice challenge how to rewrite such a system. One way would be to build a parallel system, identify all inputs and duplicate them to the parallel solution and then compare the outputs; in case of discrepancies fix the erroneous system. Once the systems produce same results (or once the new system produces better results than the old one) you just switch the systems.
The rewrite doesn't have to be complete; it can (should) be done in pieces of course.
The big issues is duplicating the system while it is still morphing in production for all the edge cases, it often feels like trying to paint a moving bus.
I would imagine this is a part of the code everyone is terrified to touch. When bugs do turn up they would be fixed by tactical 'if' statements. It would be a shit pile of hacks built up over many years.
Can add anecdotal evidence that people are afraid of changing a company's billing system as it may result in bills being generated that are significantly different as to either raise client or internal concerns, so best to leave it alone, right?
