Fictional artist Feofan Kopytto, who was immortalized in "The Little Golden Calf" [0] used oats and other cereals for his paintings. On the back of the book authors' talent it became customary to refer to his artistic endeavours as "charlatanism". Having internalized it through my Soviet upbringing, I struggle to see why the same wouldn't apply to the art being discussed. LLM kindly helped me to generate hypothetical ad copy for Kopytto in the same style [1] - I honestly see no reason to not relate to both with the same reverence (or, rather, lack thereof!). I'd appreciate a human explanation (re: why?), if anyone has a minute or two. It would help me (and maybe others) to guide understanding why AI slop of all kinds may or may not deserve the same treatment as intent-driven human "output".
Interestingly, I think this HN topic is very relevant to understanding of contemporary LLM hype, as it illustrates the power of language (and propensity of human mind) to create an appearance of substance and meaning even where there is absolute emptiness (or, worse, manipulative fraud) underneath.
I am in agreement with many commenters here (https://news.ycombinator.com/item?id=47158240, https://news.ycombinator.com/item?id=47158573 and others) that this article is a clear illustration of failure on part of AI to capture the structure of material in a useful way. As addressed in the article, the effect is very visible in visual space, 3D modeling. I would argue it is very much present in LLM space too, just less prominent due to certain properties of the medium - text-based language. I also believe the effect is fundamental, rooted in the design of those models.
I'll leave here the note I've written down recently, while thinking about this fundamental limitation.
- The relationship between sentient/human thinking and its expression ("language") is similar to the one between abstract/"vector" image specification and its rendered form (which is necessarily pixel-based/rasterised)
- "Truly reasoning" system operates in the abstract/"vector" space, only "rendering" into "raster" space for communication purposes. Today's LLMs, by their natural design, operate entirely in the "raster" space of (linguistic) "tokens". But from outside point of view the two are indistiguishable, superficially.
- Today's LLMs is a brute force mechanism, made possible by availability of sheer computing power and ample training material.
- The whole premise of LLMs ("Large" and "Language" being load-bearing words here) is that they completely bypass the need to formalize the "vector" part, conceptualize in useful manner. I call it "raster-vector impedance".
- Even if not formalized, it can be said that internal "structures" that form within LLM somehow encode/capture ("isomorphic to" is the word I like to use) the semantics ("vector"). I believe the same can be said about "computer vision" ML systems which learn to classify images after being fed billions of them.
- However, I believe that, by nature, such internal encoding is necessarily incomplete and maybe even incorrect.
- Despite the above, LLM can still be a useful tool in many domains. I think language translation is a task that can be very successfully performed without necessarily "decoding" the emerging underlying structures. I.e. a sentence in source language can be mapped onto a region of latent space; an isomorphic region of latent space based on target language can be used to produce an output in the target language which will be representative of an equivalent meaning, from human perspective. All without explicit conceptual decoding of underlying token weight matrices. "Black-box" translation, so to speak. I am amazed (and disturbed, and horrified too!) that producing a viable code in a programming language from casual natural language prompt turned out to be a subset of general translation task, largely. Well, at least on lower levels.
- To me it is intuitive that such design (brute-force transforms of "rasterized" data instead of explicitly conceptualizing it into "vector" forms) is very limited and, essentially, a dead-end.
This is a very useful take, thank you. Really helped me to adjust my mental model without "antropomorphising" the machinery. Upvoted.
If I may, I would re-phrase/expand the last sentence of yours in a way that makes it even more useful for me, personally. Maybe it could help other people too. I think it is fair to say that in presence of hints like "Pretend you are X" or "Take a deeper look" the inference mechanism (driven by it's training weights, and now influenced by those hints via "attention math") is not "satisfied" until it pulls more relevant tokens into "working context" ("more" and "relevant" being modulated by the particular hint).
Upvoted, as it basically 99% matches my own thinking. Very well said. But I, personally, would not predict a breakthrough in this direction in the next 2-5 years, as there is no pathway from current LLM tech to "true reasoning". In my mental model LLM operates in "raster space" with "linguistic tokens" being "rasterization units". For "true reasoning" an AI entity has to operate fluently in "vector space", so to speak. LLM can somewhat simulate "reasoning" to a limited degree, and even that it only does with brute force - massive CPU/GPU/RAM resources, enormous amount of training data and giant working contexts. And still, that "simulation" is incomplete and unverifiable.
I would argue that the research needed to enable such "vector operation" is nowhere near the stage to come to fruition in the next decade. So, my prediction is, maybe, 20-50 years for this to happen, if not more.
Wow, sounds so familiar! I've once had to argue precisely against this very conclusion - "you saved us once in emergency, now you're bound to do it again".
Wrote to my management: "It is, by all means, great when a navigator is able to take over an incapacitated pilot and make an emergency landing, thus averting the fatality. But the conclusion shouldn't be that navigators expected to perform more landings or continue to be backup pilots. Neither it should be that we completely retrain navigators as pilots and vice versa. But if navigators are assigned some extra responsibility, it should be formally acknowledged by giving them appropriate training, tools and recognition. Otherwise many written-off airplanes and hospitalized personnel would ensue."
For all I know the only thing this writing might have contributed to was increased resentment by management.
Like a proverbial broken clock which shows correct time twice per 24 hours, AI may "show correct time" for 99% of prompts, but doesn't deserve any more trust.
> Pat yourself on the back and mutter “you need to be skilled and lucky to work here”
It would be absolutely amazing if employers and recruiters finally were doing exactly this. We are in this dead end precisely because everyone is under false illusion that their pool of candidates has a hidden gem outshining everybody else in existence, and they absolutely need to sift through the whole pool to find this gem. As a result, all pools are never exhausted and only ever spreading, with more and more desperate people sucked into multiples of them.
Your take is very sensible and I agree with it 100%, but the reality is that (by my assessment) it is absolutely not present in the wall of ATS filters one's job application is up against. I've sent hundred of CV/cover letters over last ten months, none of them are touched by LLM. Most cover letters I manually tailored to re-frame in line with job ad - where I cared a lot, some I just made with my generic template - still manually - where I couldn't be bothered to care. Invariably I either received no response at all, or for remaining 10% I received a generic rejection email, identically worded and styled in almost all cases.
Here it is, if you are curious:
"Thank you for your interest in the <position> position at <company> in <country>. Unfortunately, we will not be moving forward with your application, but we appreciate your time and interest in <company>."
The Resume I am sending out is just an evolution of one that worked very well for me for 25+ years. The roles, as far as I am able to see, are 80%-95% keyword match, with the non-matched keywords being exceedingly superficial. Yes, I haven't listed "blob storage", but guess what else I have used but haven't listed: "semicolon", "variable declaration" and "for-loops". Yet in this day and age one seems to be punished for not doing so.
I am very principled in not letting any AI anywhere close to my CV, because I think the usefulness of signal it conveys rests solely on it being addressed to and read by human, hence it has to be fully authored and tailored by human too. But these days this idea has completely flipped. Desperate times call for desperate measures. Standing by principles could lead to literal dying. Personally, I made peace with dying, but I cannot allow my family to go homeless. As such, I don't see it below me to go down the path of mass-blasting heavily over-tailored Resumes. If it bumps my chances from 0.05% to 0.2%, that's a four-fold increase that may be the difference between, literally, life and death. The organic job search with my natural skills and authentic ways of presentation I relied on for twenty years is dead.
Interestingly, I think this HN topic is very relevant to understanding of contemporary LLM hype, as it illustrates the power of language (and propensity of human mind) to create an appearance of substance and meaning even where there is absolute emptiness (or, worse, manipulative fraud) underneath.
- [0] https://en.wikipedia.org/wiki/The_Little_Golden_Calf
- [1] https://pastebin.com/j7wsXQxe
reply