What is the difference between a google map style application that shows pixels that are "right" for a road, and pixels that are "wrong" for a road?
A pixel is a pixel, colors cannot be true or false. The only way we can say a pixel is right or wrong in google maps is whether the act of the human utilizing that pixel to understand geographical information results in the human correctly navigating the road.
In the same way, an LLM can tell me all kinds of things, and those things are just words, which are just characters, which are just bytes, which are just electrons. There is no truth or false value to an electron flowing in a specific pattern in my CPU. The real question is what I, the human, get out of reading those words, if I end up correctly navigating and understanding something about the world based on what I read.
Unfortunately, we do not want LLMs to tell us "all sorts of things," we want them to tell us the truth, to give us the facts. Happy to read how this is the wrong way to use LLMs, but then please stop shoving them into every facet of our lives, because whenever we talk about real-life applications of this tech it somehow is "not the right fit".
> we want them to tell us the truth, to give us the facts.
That's just one use case out of many. We also want it to tell stories, make guesses, come up with ideas, speculate, rephrase, ... We sometimes want facts. And sometimes it's more efficient to say "give me facts" and verify the answer then to find the facts yourself.
I think the impact of LLMs is both overhyped underestimated. The overhyping is easy to see: people predicting mass unemployment, etc., when this technology reliably fails very simple cognitive tasks and has obvious limitations that scale will not solve.
However, I think we are underestimating the new workflows this tech will enable. It will take time to search the design space and find where the value lies, as well as time for users to adapt to a different way of using computers. Even in fields like law where correctness is mission-critical, I see a lot of potential. But not from the current batch of products that are promising to replace real analytical work with a stochastic parrot.
That's a round peg in a square hole. As ive seen them called elsewhere today, these "plausible text generators" can create a pseudo facsimile of reasoning, but they don't reason, and they don't fact check. Even when they use sources to build consensus, its more about volume than authoritativeness.
I was watching the show, 3 Body Problem, and there was a great scene where a guy tells a woman to double check another man’s work. Then goes to the man and tells him to triple check the woman’s work. MoE seems to work this way, but maybe we can leverage different models that have different randomness and maybe we can get to a more logical answer.
We have to start thinking about LLM hallucination differently. When it’s follows logic correctly and provides factual information, that is also a hallucination, but one that fits our flow of logic.
Sure, but if we label the text as “factually accurate” or “logically sound” (or “unsound”) etc., then we can presumably greatly increase the probability of producing text with targeted properties
What on Earth makes you think that training a model on all factual information is going to do a lick in terms of generating factual outputs?
At that point, clearly our only problem has been we've done it wrong all along by not training these things only on academic textbooks! That way we'll only probabilistically get true things out, right? /s
> The real question is what I, the human, get out of reading those words
So then you agree with the OP that an LLM is not intelligent in the same way that Google Maps is not intelligent? That seems to be where your argument leads but you're replying in a way that makes me think you are disagreeing with the op.
I guess I am both agreeing and disagreeing. The exact same problem is true for words in a book. Are the words in a lexicon true, false, or do they not have a truth value?
The words in the book are true or false, making the author correct or incorrect. The question being posed is whether the output of an LLM has an "author," since it's not authored by a single human in the traditional sense. If so, the LLM is an agent of some kind; if not, it's not.
If you're comparing the output of an LLM to a lexicon, you're agreeing with the person you originally replied to. He's arguing that an LLM is incapable of being true or false because of the manner in which its utterances are created, i.e. not by a mind.
So only a mind is capable of something making signs that are either true or false? Is a properly calibrated thermometer that reads "true" if the temperature is over 25C incapable of being true? But isn't this question ridiculous in the first place? Isn't a mind required to judge whether or not something is true, regardless of how this was signaled?
Read again; I said he’s arguing that the LLM (i.e. thermometer in your example) is the thing that can’t be true or false. Its utterances (the readings of your thermometer) can be.
This would be unlike a human, who can be right or wrong independently of an utterance, because they have a mind and beliefs.
I’ll cut to the chase. You’re hung up on the definition of words as opposed to the utility of words.
That classical or quantum mechanics are at all useful depends on the truthfulness of their propositions. If we cared about the process then we let the non-intuitive nature of quantum mechanics enter into the judgement of the usefulness of the science.
The better question to ask is if a tool, be it a book, a thermometer, or an LLM are useful. Error rates affect utility which means that distinctions between correct and incorrect signals are more important than attempts to define arbitrary labels for the tools themselves.
You’re attempting to discount a tool based on everything other than the utility.
Ah okay I understand, I think. So basically that's solipsism applied to a LLM?
I think that's taking things a bit too far though. You can define hallucination in a more useful way. For instance you can say 'hallucination' is when the information in the input doesn't make it to the output. It is possible to make this precise, but it might be impractically hard to measure it.
An extreme version would be a En->FR translation model that translates every sentence into 'omelette du fromage'. Even if it's right the input didn't actually affect the output one bit so it's a hallucination. Compared to a model that actually changes the output when the input changes it's clearly worse.
Conceivably you could check if the probability of a sentence actually decreases if the input changes (which it should if it's based on the input), but given the nonsense models generate at a temperature of 1 I don't quite trust them to assign meaningful probabilities to anything.
No, your constant output example isn’t what people are talking about with “hallucination.” It’s not about destroying information from the input, in the sense that it you asked me a question and I just ignored you, I’m not in general hallucinating. Hallucinating is more about sampling from a distribution which extends beyond what is factually true or actually exists, such as citing a non-existent paper, or inventing a historical figure.
> It's offering support for the claim that LLMs hallucinate 100% of the time, even when their hallucinations happen to be true.
Well this makes the term "hallucinate" completely useless for any sort of distinction. The word then becomes nothing more than a disparaging term for an LLM.
Not really. It distinguishes LLM output from human output even though they look the same sometimes. The process by which something comes into existence is a valid distinction to make, even if the two processes happen to produce the same thing sometimes.
It makes sense to do so in the same way that it’s useful to distinguish quantum mechanics from classical mechanics, even if they make the same predictions sometimes.
A proposition of any kind of mechanics is what can be true or false. The calculations are not what makes up the truth of a proposition, as you’ve pointed out.
But then again, that road, according to a neighboring country who thinks that it’s their land and it isn’t a road. Depending on what country you’re in, it makes a difference
> The real question is what I, the human, get out of reading those words, if I end up correctly navigating and understanding something about the world based on what I read.
No, the real question is how you will end up correctly navigating and understanding something about the world from a falsehood crafted to be optimally harmonious with the truths that happen to surround it?
A pixel (in the context of an image) could be “wrong” in the sense that its assigned value could lead to an image that just looks like a bunch of noise. For instance, suppose we set every pixel in an image to some random value. The resulting would look like total noise and we humans wouldn’t recognized it as a sensible image. By providing a corpus of accepted images, we can train a model to generate images (arrays of pixels) which look like images and not, say, random noise. Now it could still generate an image of some place or person that doesn’t actually exist, so in that sense the pixels are collectively lying to you.
A pixel is a pixel, colors cannot be true or false. The only way we can say a pixel is right or wrong in google maps is whether the act of the human utilizing that pixel to understand geographical information results in the human correctly navigating the road.
In the same way, an LLM can tell me all kinds of things, and those things are just words, which are just characters, which are just bytes, which are just electrons. There is no truth or false value to an electron flowing in a specific pattern in my CPU. The real question is what I, the human, get out of reading those words, if I end up correctly navigating and understanding something about the world based on what I read.