Readit News logoReadit News
walkabout commented on 'Attention is all you need' coauthor says he's 'sick' of transformers   venturebeat.com/ai/sakana... · Posted by u/achow
visarga · 2 months ago
It seems unfair to call out LLMs for "spam, scams, propaganda, and astroturfing." These problems are largely the result of platform optimization for engagement and SEO competition for attention. This isn't unique to models; even we, humans, when operating without feedback, generate mostly slop. Curation is performed by the environment and the passage of time, which reveals consequences. LLMs taken in isolation from their environment are just as sloppy as brains in a similar situation.

Therefore, the correct attitude to take regarding LLMs is to create ways for them to receive useful feedback on their outputs. When using a coding agent, have the agent work against tests. Scaffold constraints and feedback around it. AlphaZero, for example, had abundant environmental feedback and achieved amazing (superhuman) results. Other Alpha models (for math, coding, etc.) that operated within validation loops reached olympic levels in specific types of problem-solving. The limitation of LLMs is actually a limitation of their incomplete coupling with the external world.

In fact you don't even need a super intelligent agent to make progress, it is sufficient to have copying and competition, evolution shows it can create all life, including us and our culture and technology without a very smart learning algorithm. Instead what it has is plenty of feedback. Intelligence is not in the brain or the LLM, it is in the ecosystem, the society of agents, and the world. Intelligence is the result of having to pay the cost of our execution to continue to exist, a strategy to balance the cost of life.

What I mean by feedback is exploration, when you execute novel actions or actions in novel environment configurations, and observe the outcomes. And adjust, and iterate. So the feedback becomes part of the model, and the model part of the action-feedback process. They co-create each other.

walkabout · 2 months ago
> It seems unfair to call out LLMs for "spam, scams, propaganda, and astroturfing." These problems are largely the result of platform optimization for engagement and SEO competition for attention.

They didn't create those markets, but they're the markets for which LLMs enhance productivity and capability the best right now, because they're the ones that need the least supervision of input to and output from the LLMs, and they happen to be otherwise well-suited to the kind of work it is, besides.

> This isn't unique to models; even we, humans, when operating without feedback, generate mostly slop.

I don't understand the relevance of this.

> Curation is performed by the environment and the passage of time, which reveals consequences.

It'd say it's revealed by human judgement and eroded by chance, but either way, I still don't get the relevance.

> LLMs taken in isolation from their environment are just as sloppy as brains in a similar situation.

Sure? And clouds are often fluffy. Water is often wet. Relevance?

The rest of this is a description of how we can make LLMs work better, which amounts to more work than required to make LLMs pay off enormously for the purposes I called out, so... are we even in disagreement? I don't disagree that perhaps this will change, and explicitly bound my original claim ("so far") for that reason.

... are you actually demonstrating my point, on purpose, by responding with LLM slop?

walkabout commented on US tariff negotiations with Canada terminated over advertisement   bbc.com/news/articles/cdj... · Posted by u/mlhpdx
tgma · 2 months ago
I agree there are incorrect information, incorrect analysis, and incorrect predictions by the electorate. What I am saying is that in aggregate, the political machine on both sides is fully incentivized with enough financial and media backing to counter the other side. It is not even inconceivable to see each individual vote for their "right" candidate for the wrong reasons. I fully acknowledge that.

In aggregate, however, I believe in the US presidential elections end up voting for their own best interests, as they see it, and even if they become unhappy with the state of the world after four years, it appears to be unlikely to find people who say they would have switched votes. If anything, they are becoming more polarized and committed to one side, thus harder to "fool." In that sense, they are not mistaken. The human experience is not a set of entirely quantifiable metrics, and being "happily-fooled" is also a human interest, as long as they don't get buyer's remorse. Lots of buyer's remorse is really the only metric that can prove the counterpoint.

What GP is saying is isomorphic to telling Apple customers "you don't know your interests and Apple is charging you too much while keeping you in the walled garden." Maybe right, maybe wrong, but who are you to judge they would have been better off with a Dell?

walkabout · 2 months ago
> In aggregate, however, I believe in the US presidential elections end up voting for their own best interests, as they see it

This is extremely close to one of the early "OK, but maybe there's a reason what we're observing at the individual level isn't so scary" hypotheses explored by political science in the latter half of the 20th century—that individually poor choices would nonetheless produce good outcomes by being in some way chaotic and the good outcomes often manifesting as attractors in that chaotic space, or something like that, or by some "wisdom of the crowds" effect that emerges in aggregate. These approaches have been found untenable despite much trying, though I think there are some limited efforts at it still under way.

HOWEVER! I think after this post I do see what you're actually getting at, which is that if people believe they voted in their own best interests ("as they see it" being key) then they may believe they did in-fact do that indefinitely, even if entirely incorrect, so long as they... well, continue to believe so.

The prisoner voting to remain a prisoner not because they don't want to be free—not because if you describe completely and in detail, leaving nothing out, the conditions they're in-fact in they tell you they would love to live that way (they claim they would hate it!), and then if you also describe free life they claim that is the outcome they would rather have, and if you carefully probe you find that it's not even for some greater-interest purpose they are voting to remain imprisoned (it's not that they believe they'd be a danger to others if free, for example), but because they believe they aren't in prison despite [gestures at their prison cell]—is voting in their own interest.

By that standard, yes, a lot more voters are voting in their own interest than may be reckoned by other standards.

walkabout commented on 'Attention is all you need' coauthor says he's 'sick' of transformers   venturebeat.com/ai/sakana... · Posted by u/achow
warkdarrior · 2 months ago
Spam detection and phishing detection are completely different than 5 years ago, as one cannot rely on typos and grammar mistakes to identify bad content.
walkabout · 2 months ago
Spam, scams, propaganda, and astroturfing are easily the largest beneficiaries of LLM automation, so far. LLMs are exactly the 100x rocket-boots their boosters are promising for other areas (without such results outside a few tiny, but sometimes important, niches, so far) when what you're doing is producing throw-away content at enormous scale and have a high tolerance for mistakes, as long as the volume is high.
walkabout commented on Counter-Strike's player economy is in a freefall   polygon.com/counter-strik... · Posted by u/perihelions
pessimizer · 2 months ago
No casino gambling. Casino gambling is not gambling. It's putting $1.00 into a machine and getting $0.80 back, but the exact refund amount after every dollar put in is arbitrary. There's absolutely no risk to the casino, the casino is not gambling. The only gambling being done is by individual bettors, and they have an expected massive loss.

I don't care very much if people gamble with each other, and expect $1 back for every $1 they put in.* But casino games and lootboxes are specifically designed for consistent losses to the house. It's simply another tax, but on the addicted, desperate, and/or innumerate. The weakest people at their weakest moments; and if we're not protecting them, the government has no purpose.

* I actually think that it is good for people who have the same wealth levels to gamble with each other, as long as the outcomes are largely random. The problem is with vigs, and with pots that get too large to cover against a house that can endlessly extend itself.

walkabout · 2 months ago
Yeah, I think you nailed it. A ban on playing against "the house" would do it. Taking a fixed amount from each pot (as at poker tables) for play among patrons would still be allowed, but slot machines wouldn't. Your solution's much better than a full ban because it wouldn't drive as much illegal betting (a problem no only because it circumvents the law, but because for gambling in particular but for any black market, really, it tends to become connected with other criminal activity)
walkabout commented on US tariff negotiations with Canada terminated over advertisement   bbc.com/news/articles/cdj... · Posted by u/mlhpdx
tgma · 2 months ago
I understand that. That is not the point though. Although, if you believe in that theory, you should reject democracy and aim for some form of aristocracy or monarchy. I don't believe that many political scientists [sic] today publish and advocate disenfranchisement, perhaps because that's not politically correct, but all that is beside the point.

My point specifically is if people are voting for someone, more often than not (at least in the US, perhaps less so elsewhere where they elect the parliament and the parliament by proxy elects the executive which induces some machinations), want that person for whatever reason and consider that person aligned with their interests even if some second-order effects are not so. They did not get "fooled" and bait-and-switched even if they later feel the performance was not great. Proof for that is you are not going to find that many who say they would have switched their votes even after the fact. Those political scientists and the GP have the arrogance and audacity to project their own interests on every single person and conclude they did not vote appropriately.

walkabout · 2 months ago
> Although, if you believe in that theory, you should reject democracy and aim for some form of aristocracy or monarchy.

Not necessarily! It means that the model of the typical voter's behavior (and of the reasons why elections go the ways they do) isn't what many conceive it to be (or hope it may be), and that democracy's weaknesses, vulnerabilities, strengths, and capabilities may in-fact be at least somewhat different from what one operating from that idealized (and apparently very wrong) model of voter behavior would expect. It could still be the best of a bad lot.

> They did not get "fooled" and bait-and-switched even if they later feel the performance was not great.

They are extremely often operating from incorrect information, either regarding facts about the state of the world, or about probable outcomes of various policies. This can include things that directly affect them (or don't) in ways that one would expect them to notice—one fun form of study that's been run a few times is to ask a population whether a tax increase or decrease that in-fact affected only a tiny sliver of the population but was the subject of substantial propagandizing and/or publicity affected them personally (this is about as direct as it gets!) and the typical result is pretty much exactly what your most-pessimistic guess would be.

Supposing that people very-often hold a bunch of incorrect beliefs about how policies affect them but are also good at voting for their own interests when it comes time to mark the ballot is probably somewhere in the category of wishful thinking—and that's assuming motivations and intentions focused on policies and their outcomes in the first place. There's less-strong but still-quite-strong evidence that, as the kids say, "vibes" are a huge factor in the outcomes of elections, even when those "vibes" come from things that even the extremely politically-ignorant ought to know have nothing much to do with, say, who the President is, like a rash of shark attacks for example. This, of course, doesn't mean that this "vibes-from-irrelevant-stuff" voting makes the difference for anywhere near as many people as incorrect information does (it almost certainly doesn't) but that it has an outsize effect on the true-swing (not self-reported swing, that's mostly bullshit) vote, which tends to consist almost entirely of so-called "low-information voters", with the result that it may not have any effect at all on most voters but elections still turn on it (one of a billion reasons FPTP voting sucks is that it amplifies the power of this effect).

I do think, separately, there are cases of rational trade-offs, of picking (say) an anti-abortion candidate who holds many other positions one dislikes because one's stake in one's position on abortion is that important. That's not the kind of thing I mean, and I don't think it's the kind of thing most people mean when they say people are making mistakes by "voting against their own interests", though the effect of such a choice may well be that one is also in these cases (consciously!) voting against one's own interests on various issues.

walkabout commented on US tariff negotiations with Canada terminated over advertisement   bbc.com/news/articles/cdj... · Posted by u/mlhpdx
FranzFerdiNaN · 2 months ago
Yes, old habits die hard, but it is slowly changing.
walkabout · 2 months ago
One may smile while sharpening a knife.
walkabout commented on US tariff negotiations with Canada terminated over advertisement   bbc.com/news/articles/cdj... · Posted by u/mlhpdx
beAbU · 2 months ago
Having grown up in an African country that has been "collapsing" for the last 30 years, and looking at the neighbouring countries that went through or is going through similar issues, I think the US has far to go before things can be regarded as "collapsed". Unfortunately such an implosion is not instant, but it's a gradual frog-boiling decline that gets harder and harder to get out of.

If you are hoping for some limit to be reached where a popular uprising will be triggered, then I would advise to not put your hopes on it.

walkabout · 2 months ago
We're on track to be a much-richer Russia. Which is a lot better than being like Russia, and also as poor as Russia. But it's a lot worse than being rich and also not like Russia.

I expect most of the pain will be from lost potential growth rather than an actual decline in real terms, and that it'll take a while for most people to realize how stagnant we've become—because line will continue to Go Up thanks to an inflation-based debt reduction strategy, plus the US is such a giant player in the global economy that our slowing way down will also slow the global economy for quite a while, until it adjusts, so we'll still seem to be doing relatively OK for potentially another decade or more.

Of course we could also derail into something even worse than Russia. Or capital flight might hit us harder and faster than I think it will (anyone who can't see a way to, or can't stomach, getting on the good side of our rulers, will want to get out so they don't lose all their shit, including possibly their lives in extreme cases)

walkabout commented on US tariff negotiations with Canada terminated over advertisement   bbc.com/news/articles/cdj... · Posted by u/mlhpdx
tgma · 2 months ago
> I’ve been waiting to see my countrymen vote in their own best interests for my entire life

This is an extremely arrogant statement to think a single individual can know the best interests of an entire country and to know they were wrong in identifying their own. To quantify this, perhaps one close proxy is to see how many people really regretted their vote after the fact, which in the context of US does not appear to be that many (even those unsatisfied with the outcome post hoc would not necessarily have voted for the opponent if given a time machine.)

Perhaps it is not trivial to have visibility into the intricacies of other people's lives and their priorities. Even harder to generalize it to tens of millions of people in a country.

walkabout · 2 months ago
Voter behavior and motivation (and knowledge of the issues, and of basic facts about their own government...) is well-studied and has been for decades. Political scientists studied it really heavily for quite a while because early results were fucking alarming (and proved to be accurate, and also not just a temporary aberration) if you're starting from a firm belief in liberal democracy and a broad franchise.

Voters, to a great extent, aren't motivated by what one might either expect or hope, nor 1/10th as well informed about the operation of their own government or the issues at stake as one might hope. It's a shit-show, so much so that it's practically miraculous that voting produces functioning governments ever, at all, and the whole thing's terribly fragile (after convincing themselves the data weren't wrong, the next step was a few decades of trying to figure out some mechanism by which this whole thing wasn't as worrisome as it seemed, which effort turned out to be based mostly on "copium", to use a modern term, and was eventually regarded as having more-or-less failed)

walkabout commented on Armed police swarm student after AI mistakes bag of Doritos for a weapon   dexerto.com/entertainment... · Posted by u/antongribok
cookiengineer · 2 months ago
I recommend rewatching the trilogy of Brazil, 12 Monkeys and Zero Theorem.

It's sadly the exact future that we are already starting to live in.

walkabout · 2 months ago
I have seen and enjoyed the first two movies but had somehow never even heard of the third one. It’s now high up my to-watch list, thanks for bringing it to my attention.
walkabout commented on Counter-Strike's player economy is in a freefall   polygon.com/counter-strik... · Posted by u/perihelions
mosselman · 2 months ago
How about no gambling at all? That would work for me.
walkabout · 2 months ago
Yeah I used to be for it on grounds of liberty but having seen a little of the actual industry it’s just purely corrosive, evil shit. It should be fought.

I’d maybe be OK with some kind of well-thought-through thing that still allowed friendly poker matches or sports brackets between people who actually know each other, but got the big money out of it. Maybe just ban corporations from having anything to do with it so limited-liability and serious investment is taken off the table? Something along those lines? But it’s also bad enough that I’d definitely vote for an outright ban if it came up. Complete switch-around for me on this topic, from where I was on it for years.

u/walkabout

KarmaCake day296October 5, 2025View Original