> I'm a bit torn. My first thought was "If the current state of the art LLMs made the mistakes it's unlikely an LLM would be able to correct them."
But, I mean, that doesn't make sense even for humans, right? 99% of the errors I make, I can easily correct myself because they're trivial. But you still have to go through the process of fixing them, it doesn't happen on its own. Like, for instance, just now I typoed "stil" and had to backspace a few letters to fix it. But LLMs cannot backspace (!), so you have to put them into a context where they feel justified in going back and re-typing their previous code.
That's why it's a bit silly to make LLM code, try to run it, see an error and immediately run to the nearest human. At least let the AI do a few rounds of "Can you spot any problems or think of improvements?" first!
But, I mean, that doesn't make sense even for humans, right? 99% of the errors I make, I can easily correct myself because they're trivial. But you still have to go through the process of fixing them, it doesn't happen on its own. Like, for instance, just now I typoed "stil" and had to backspace a few letters to fix it. But LLMs cannot backspace (!), so you have to put them into a context where they feel justified in going back and re-typing their previous code.
That's why it's a bit silly to make LLM code, try to run it, see an error and immediately run to the nearest human. At least let the AI do a few rounds of "Can you spot any problems or think of improvements?" first!