In aggregate, however, I believe in the US presidential elections end up voting for their own best interests, as they see it, and even if they become unhappy with the state of the world after four years, it appears to be unlikely to find people who say they would have switched votes. If anything, they are becoming more polarized and committed to one side, thus harder to "fool." In that sense, they are not mistaken. The human experience is not a set of entirely quantifiable metrics, and being "happily-fooled" is also a human interest, as long as they don't get buyer's remorse. Lots of buyer's remorse is really the only metric that can prove the counterpoint.
What GP is saying is isomorphic to telling Apple customers "you don't know your interests and Apple is charging you too much while keeping you in the walled garden." Maybe right, maybe wrong, but who are you to judge they would have been better off with a Dell?
This is extremely close to one of the early "OK, but maybe there's a reason what we're observing at the individual level isn't so scary" hypotheses explored by political science in the latter half of the 20th century—that individually poor choices would nonetheless produce good outcomes by being in some way chaotic and the good outcomes often manifesting as attractors in that chaotic space, or something like that, or by some "wisdom of the crowds" effect that emerges in aggregate. These approaches have been found untenable despite much trying, though I think there are some limited efforts at it still under way.
HOWEVER! I think after this post I do see what you're actually getting at, which is that if people believe they voted in their own best interests ("as they see it" being key) then they may believe they did in-fact do that indefinitely, even if entirely incorrect, so long as they... well, continue to believe so.
The prisoner voting to remain a prisoner not because they don't want to be free—not because if you describe completely and in detail, leaving nothing out, the conditions they're in-fact in they tell you they would love to live that way (they claim they would hate it!), and then if you also describe free life they claim that is the outcome they would rather have, and if you carefully probe you find that it's not even for some greater-interest purpose they are voting to remain imprisoned (it's not that they believe they'd be a danger to others if free, for example), but because they believe they aren't in prison despite [gestures at their prison cell]—is voting in their own interest.
By that standard, yes, a lot more voters are voting in their own interest than may be reckoned by other standards.
Therefore, the correct attitude to take regarding LLMs is to create ways for them to receive useful feedback on their outputs. When using a coding agent, have the agent work against tests. Scaffold constraints and feedback around it. AlphaZero, for example, had abundant environmental feedback and achieved amazing (superhuman) results. Other Alpha models (for math, coding, etc.) that operated within validation loops reached olympic levels in specific types of problem-solving. The limitation of LLMs is actually a limitation of their incomplete coupling with the external world.
In fact you don't even need a super intelligent agent to make progress, it is sufficient to have copying and competition, evolution shows it can create all life, including us and our culture and technology without a very smart learning algorithm. Instead what it has is plenty of feedback. Intelligence is not in the brain or the LLM, it is in the ecosystem, the society of agents, and the world. Intelligence is the result of having to pay the cost of our execution to continue to exist, a strategy to balance the cost of life.
What I mean by feedback is exploration, when you execute novel actions or actions in novel environment configurations, and observe the outcomes. And adjust, and iterate. So the feedback becomes part of the model, and the model part of the action-feedback process. They co-create each other.
They didn't create those markets, but they're the markets for which LLMs enhance productivity and capability the best right now, because they're the ones that need the least supervision of input to and output from the LLMs, and they happen to be otherwise well-suited to the kind of work it is, besides.
> This isn't unique to models; even we, humans, when operating without feedback, generate mostly slop.
I don't understand the relevance of this.
> Curation is performed by the environment and the passage of time, which reveals consequences.
It'd say it's revealed by human judgement and eroded by chance, but either way, I still don't get the relevance.
> LLMs taken in isolation from their environment are just as sloppy as brains in a similar situation.
Sure? And clouds are often fluffy. Water is often wet. Relevance?
The rest of this is a description of how we can make LLMs work better, which amounts to more work than required to make LLMs pay off enormously for the purposes I called out, so... are we even in disagreement? I don't disagree that perhaps this will change, and explicitly bound my original claim ("so far") for that reason.
... are you actually demonstrating my point, on purpose, by responding with LLM slop?