One thing that keeps me up at night is that the human genome is only 3 giga base pairs, of which only a fraction encodes the design of our brains — and quite inefficiently at that, through layers of indirections.
That’s sufficient information to produce a system that can learn to think like us! Not just learn but efficiently, with far less input data needed than any current LLM. Literally just a couple of decades of video and audio, only a small fraction of that text!
From what I can tell, the human brain achieves this through scale alone. There’s nothing else that can explain the observed learning capability. The genome is too small to encode learned weights, and we don’t clone our parents’s brains in development.
It’s possible, and a meat computer can do it. Replicating this in silicon is just a matter of time, and it might require only scale and nothing else.
We have sensors for vision, sound, taste, temperature, texture, etc that are constantly observing the world and affecting not only our current behavior but changing us in real time.
You can feed an LLM of typical size (~1 TB) all the video and audio you can find, but it won't turn into a human. I suspect but can't prove that even if you wired up a bunch of other sensors, gave it a robot body, and let it "explore" the world on its own, that would still be woefully insufficient.
A gazelle just days(!) old can control its body and four legs sufficiently well to outrun a cheetah. This is a complex motor-control loop involving all of its senses. Compare that to Tesla's autopilot training system, which uses many millions of hours of training data and still struggles to move a car... slowly. The equivalent would be a training routine that can take just a handful of days of footage and produce an AI that can win a car race.
There's something magical about neural networks when scaled up to brain sizes. From what I gather, there's little else encoded in the genome except for the high-level pattern of wiring, the equivalent to the PyTorch model configuration.
How many hours or actually years of evolution have been needed until reaching walking capability? If first life is believed to have happened 4 billion years ago, and first walking animals started 450 million years ago during the siluariab period, that’s around 3,5 billion years.
Meat computers physically change when they learn. They don't separate data from compute hardware, the two are the same thing, so they scale really well. Silicon doesn't currently have that capability.
That’s sufficient information to produce a system that can learn to think like us! Not just learn but efficiently, with far less input data needed than any current LLM. Literally just a couple of decades of video and audio, only a small fraction of that text!
From what I can tell, the human brain achieves this through scale alone. There’s nothing else that can explain the observed learning capability. The genome is too small to encode learned weights, and we don’t clone our parents’s brains in development.
It’s possible, and a meat computer can do it. Replicating this in silicon is just a matter of time, and it might require only scale and nothing else.