I also should have added that the ability to self explain when COT was in use only goes as deep as the COT, as soon as you probe deeper such that the content of the COT requires explanation the LLM is back in the realm of purely making stuff up again.

A non-hallucinated answer could only recount the COT and beyond that it would only be able to answer "Instinct."-- sure the LLM's response has reasoning hidden inside it, but that reasoning is completely inaccessible to the LLM.