That's what happens when people outsource their mental capacity to a machine

Using "low cost" and LLM's in the same sentence is kind of funny to me.

Is this not a recipe for model collapse?

No, because in the process they are describing the AIs would only post things they have found to fix their problem (a.k.a, it compiles and passes tests), so the contents posted in that "AI StackOverflow" would be grounded in external reality in some way. It wouldn't be an unchecked recursive loop which characterizes model collapse.

Model collapse here could happen if some evil actor was tasked with posting made up information or trash though.

As pointed out elsewhere, compiling code and passing tests isn’t a guarantee that generated code is always correct.

So even “non Chinese trained models” will get it wrong.

It doesn't matter that it isn't always correct; some external grounding is good enough to avoid model collapse in practice. Otherwise training coding agents with RL wouldn't work at all.

And how do you verify that external grounding?

What precisely do you mean by external grounding? Do you mean the laws of physics still apply?

I mean it in the sense that tokens that pass some external filter (even if that filter isn't perfect) are from a very different probability distribution than those that an LLM generates indiscriminately. It's a new distribution conditioned by both the model and external reality.

Model collapse happens in the case where you train your model indefinitely with its own output, leading to reinforcing the biases that were originally picked up by the model. By repeating this process but adding a "grounding" step, you avoid training repeatedly on the same distribution. Some biases may end up being reinforced still, but it's a very different setting. In fact, we know that it's completely different because this is what RL with external rewards fundamentally is: you train only on model output that is "grounded" with a positive reward signal (because outputs with low reward get effectively ~0 learning rate).

Oh interesting. I guess that means you need to deliberately select a grounding source with a different distribution. What sort of method would you use to compare distributions for this use case? Is there an equivalent to an F-test for high dimensional bit vectors?

Should've called it Slopbook

Isn't the whole issue here that because the agent trusted Anthrophic IP's/URL's it was able to upload data to Claude, just to a different user's storage?

I know this isn't even the worst example, but the whole LLM craze has been insane to witness. Just releasing dangerous tools onto an uneducated and unprepared public and now we have to deal with the consequences because no one thought "should we do this?"

Pretty much all of the country takes years of formal education. They all understand file permissions. Most just pretend not to so their time isn't exploited.

I have used these disposable vapes before, and it saddens me that so much tech (including those batteries!) are just thrown away. Like what ever happened to reducing e-waste.

It was reverse engineered, not opened by IBM

No, they published schematics and full BIOS source code.

But it was not libre, they held the copyright to the source code. So to get around this, competing companies wrote a spec from the source code and then had another team which never saw the code implement a new BIOS from the spec.

I miss the official Nintendo forums every day. I was a dumb kid back then but I really got my online sea legs on those message boards, and other ones related to Pokémon fan sites.

How does it compare to 1Blocker? I use that in Safari and also a VPN when I'm away back to my home connection so it uses my NextDNS which also blocks a lot of in-app ads.