Readit NewsThey're just highly scaled up versions of their smaller curve fitting cousins. For those we can even make pretty visualizations that show exactly what is happening as the network "learns".
I don't mean "we can see parts of the brain light up", I mean "we can see the cuts and bends each ReLU is doing to the input".
We built these things, we know exactly how they work. There's no surprise beyond just how good prediction accuracy gets with a big enough model.
Deep Neural Networks are also a very different architecture from what is going on in our brains (which work more like Spiking Neural Networks) and our brains don't do backpropagation, so you can't even make direct comparisons.
My best argument that insights from neuroscience will transfer to neural networks, and vice versa:
For sufficiently complex phenomena (e.g., language), there should only be one reasonably efficient solution to the problem, and small variations on that solution. So there should be some reversible mapping between any two tractable solutions to the problem that is pretty close to lossless, provided both solutions actually solve the problem.
And, yeah, the main advantage of neural networks is that they're white-box. You can also control your experiments in a way you can't in the real world.
Policy gradient approaches in RL are just supervised learning, when viewed through some lenses. You can search for karpathys more fleshed out argument for the same, I'm on mobile jow.
So you're amplifying existing trajectories in the model by feeding the model's outputs back to itself, but only when those outputs solve a problem.
This elides the kl penalty and the odd group scoring, which are the same in the limit but vastly more efficient in practice.
(GPUs are generally much more cost effective and energy efficient than CPU if the solution maps to both architectures. Anthropic certainly caches the KV-cache of their 24k token system prompt.)
If we can get past this specific thing we can perhaps have a fact-based conversation about what's going on with power use in ai or tech.
Things that are not true about neural networks and LLMs:
1. They are not stochastic parrots. Statistics can be used to judge model performances, as with any type of model whatsoever.
But that doesn’t make a model statistical. Neural network components are not statistically driven, they are gradient driven. They don’t calculate statistics, they learn topological representations of relationships.
2. LLMs are not just doing word prediction or text completion. Yes, that is the basic task they are trained on, but the “just” (that is often stated or implied) trivializes what the network actually has to learn to perform well.
Task type, and what must be learned to achieve success at that task, are two entirely different things.
To predict the kinds of human reasoning documented in writing in training sets, requires that kind of reasoning be learned. Not just some compressed generalization of people’s particular responses.
Simple proof that LLMs are not just compressing a lot of human behavior comes easily. Just ask an LLM to do something involving several topics unlikely to have been encountered together before, and their one-shot answer might not be God’s word on the issue, but is a far cry from the stumbling that good mimic could ever do. (Example task, ask for a Supreme Court brief to argue for rabbits rights based on sentient animal and native rights, with serious arguments, but written in Dr. Seuss prose by James Bond.)
3. LLMs do reason. Not like us or as well as us. But also, actually better than us in some ways.
LLMs are far superior to us at very wide, somewhat shallow reasoning.
They are able to one-shot weave together information from tens of thousands of disparate topics and ideas, on demand.
But they don’t notice their own blind spots. They miss implications. Those are things humans do quickly in continuous narrower deeper reasoning cycles.
They are getting better.
And some of their failures should be attributed to the strict constraint of one/few-shot(s) to well written responses, that we put on them.
We don’t hold humans to such strict standards. And if we did, we would also make a lot more obvious errors.
Wide & shallow reasoning in ways humans can’t match, is not a trivial success.
4. LLMs are very creative.
As with their reasoning, not like us: very wide in terms of instantly and fluidly weaving highly disparate information together. Somewhat shallow (but gaining) in terms of iteratively recognizing and self-correcting their own mistakes and oversights.
See random original disparate topic task above.
First, spend several hours performing that task oneself. Or take a day or two if need be.
Then, give the task to an LLM and wait a few seconds.
Compare. Ouch.
—
TLDR; somewhat off topic and high level, but when trying to understand models it helps to avoid seemingly endlessly repeated misunderstandings.
I think it's true that models are statistical, inasmuch as P(A|B) where B is the prior sequence is what the loss is computing, and that's statistical. It's just computing that function in an absurdly complex way, which involves creating topological representations of relationships, etc.
I agree that "just" autocomplete implies the wrong thing. It turns out autocomplete is amazing if you scale it up.
I think it's true that they reason and area creative but these are really hard points because people mean subtly different things when saying "reason" and "creative".