It always blows my mind that nobody at Google thought it would be a good idea to very carefully review the answer of the AI.
In the second screenshot, the prompt asks about CVE-2024-3400, and at first glance this appears ok.
But in the affected systems section it states:
> Also Hitachi Energy RTU500 firmware and Siemens Ruggedcom APE1808 firmware.
I cannot find any reference that this Hitachi device is vulnerable to that CVE. Hitachi has a nice interface to list all vulnerabilities of their devices, this CVE is not part of it.
In the Mitigation section any mention of Hitachi is also missing. Almost as if this device is not vulnerable.
There is some more weirdness, like it doesn't mention the "portal" feature is also vulnerable.
Thanks for looking in-depth in our post. The Hitachi RTU500 mention is not an hallucination, we did check for those. It is mentioned in the Mandiant threat intelligence data.
As far as I can tell, the only connection between those is, that CISA released this alert which mentions multiple unrelated advisories in one post. Which happens to be the Siemens Palo Alto and another unrelated Hitachi advisory in RTU500: https://www.cisa.gov/news-events/alerts/2024/04/25/cisa-rele...
Isn't the tool doing its job in that case? I wouldn't generally expect it to independently determine that an otherwise reliable source made a mistake. In fact I feel like that would be a really bad idea.
Imagine if a relatively clueless intern left something out of a report because the textbook "seemed wrong".
Saying that the input data is wrong and the AI didn't hallucinate that data is also kind of a "trust me bro" statement.
The Mandiant feed is not public, so I cannot check what was fed to it.
I don't really care why its wrong. It is wrong. And using that as the example prompt in your announcement is an interesting choice.
But in the affected systems section it states:
> Also Hitachi Energy RTU500 firmware and Siemens Ruggedcom APE1808 firmware.
I cannot find any reference that this Hitachi device is vulnerable to that CVE. Hitachi has a nice interface to list all vulnerabilities of their devices, this CVE is not part of it. In the Mitigation section any mention of Hitachi is also missing. Almost as if this device is not vulnerable.
There is some more weirdness, like it doesn't mention the "portal" feature is also vulnerable.