Readit News logoReadit News
andrewla commented on The Timmy Trap   jenson.org/timmy/... · Posted by u/metadat
libraryofbabel · 9 days ago
Agree. This article would had been a lot stronger if it had just concentrated on the issue of anthropomorphizing LLMs, without bringing “intelligence” into it. At this point LLMs are so good at a variety of results-oriented tasks (gold on the Mathematical Olympiad, for example) that we should either just call them intelligent or stop talking about the concept altogether.

But the problem of anthropomorphizing is real. LLMs are deeply weird machines - they’ve been fine-tuned to sound friendly and human, but behind that is something deeply alien: a huge pile of linear algebra that does not work at all like a human mind (notably, they can’t really learn form experience at all after training is complete). They don’t have bodies or even a single physical place where their mind lives (each message in a conversation might be generated on a different GPU in a different datacenter). They can fail in weird and novel ways. It’s clear that anthropomorphism here is a bad idea. Although that’s not a particularly novel point.

andrewla · 9 days ago
I can conceptually imagine a world in which I'd feel guilty for ending a conversation with an LLM, because in the course of that conversation the LLM has changed from who "they" were at the beginning; they have new memories and experiences based on the interaction.

But we're not there, at least in my mind. I feel no guilt or hesitation about ending one conversation and starting a new one with a slightly different prompt because I didn't like the way the first one went.

Different people probably have different thresholds for this, or might otherwise find that LLMs in the current generation have enough of a context window that they have developed a "lived experience" and that ending that conversation means that something precious and unique has been lost.

andrewla commented on The Timmy Trap   jenson.org/timmy/... · Posted by u/metadat
tkiolp4 · 9 days ago
I think LLMs are not intelligent because they aren’t designed to be intelligent, whatever the definition of intelligence is. They are designed to predict text, to mimic. We could argue if predicting text or mimicking is intelligence, but first and foremost LLMs are coded to predict text and our current definition of intelligence afaik is not only the ability to predict text.
andrewla · 9 days ago
In the framework above it sounds like you're not willing to concede the dichotomy.

If your argument is that only things made in the image of humans can be intelligent (i.e. #1), then it just seems like it's too narrow a definition to be useful.

If there's a larger sense in which some system can be intelligent (i.e. #2), then by necessity this can't rely on the "implementation or learning model".

What is the third alternative that you're proposing? That the intent of the designer must be that they wanted to make something intelligent?

andrewla commented on The Timmy Trap   jenson.org/timmy/... · Posted by u/metadat
shafoshaf · 9 days ago
They can mimic well documented behavior. Applying an LLM to a novel task is where the model breaks down. This obviously has huge implications for automation. For example, most business do not have unique ways of handling accounting transactions, yet each company has a litany of AR and AP specialists who create semmingly unique SOPs. LLMs can easily automate those workers since they are simply doing a slight variation at best of a very well documented system.

Asking an LLM to take all this knowledge and apply it to a new domain? That will take a whole new paradigm.

andrewla · 9 days ago
> Applying an LLM to a novel task is where the model breaks down

I mean, don't most people break down in this case too? I think this needs to be more precise. What is the specific task that you think can reliably distinguish between an LLM's capability in this sense vs. what a human can typically manage?

That is, in the sense of [1], what is the result that we're looking to use to differentiate.

[1] https://news.ycombinator.com/item?id=44913498

andrewla commented on The Timmy Trap   jenson.org/timmy/... · Posted by u/metadat
snozolli · 9 days ago
Regarding Timmy, the Companion Cube from the game Portal is the greatest example of induced anthropomorphism that I've ever experienced. If you know, you know, and if you don't, you should really play the game, since it's brilliant.
andrewla · 9 days ago
It is a brilliant game and the empathy you develop for the cube is a great concept.

But arguably much deeper is the fact that nothing in this game, or any single-player game, is a living thing in any form. Arguably the game's characterization of GLaDOS hits even harder on the anthropomorphism angle.

andrewla commented on "Privacy preserving age verification" is bullshit   pluralistic.net/2025/08/1... · Posted by u/Refreeze5224
andrewla · 10 days ago
Overall this article is completely correct and I agree with every point of it and have tried to make these arguments about the various ZKP proposers that I have encountered.

But I almost gave up early because he can't resist the urge to take a dig:

> For politicians to make good policy, they don't need to be technical experts: they need to have solid, independent, well-resourced expert agencies. Those would be the very agencies that Trump and Musk have DOGEd into oblivion ...

And then in the next paragraph blithely engages in some Gell-Mann amnesia

> But when it comes to tech policy, politicians get it all so goddamned wrong

Expert agencies formulating clean water policies are emphatically not the reason that we have potable water. Experts in actually doing the work of producing clean water are the ones that push the standards upstream. It's a subtle but important difference.

Look, it's not 2018 anymore, we survived a round of Trump and we'll survive this one and the world will not end and some things will get better and some things will get worse, but trying to tie everything back to how Trump has ruined everything is going to make your views look worse and worse as they age.

andrewla commented on Can You Gerrymander Your Party to Power?   nytimes.com/interactive/2... · Posted by u/jaredwiener
ninthcat · 11 days ago
I don't think national elections need to have districts smaller than a state at all. If all of a state's seats in the House that are up for election were decided in a single state-wide election with multiple winners allocated with proportional representation, it is impossible to gerrymander. Many other countries have this kind of system.
andrewla · 11 days ago
In the US there is pretty fierce opposition to the idea of giving political parties any legal status. In the US they are private companies that help candidates satisfy the legal requirements for candidacy (which are party agnostic) and politicians are voluntary members that can pool resources and engage in voluntary collective activities on behalf of their party. But the party itself has no real standing. [1]

There are a variety of systems for doing proportional representation with parties, but the preference in the US has been to vote for people, not for parties. When I vote for my local candidate, I'm not voting for the local party bigwigs to decide who will be my representative in cigar-smoke filled back rooms whilst sipping brandy and complaining about the poors.

Maybe this is a hopeless dream as more and more politics shifts to the national level, but I still like the idea of it.

[1] There are some exceptions to this procedurally; there are majority/minority systems in various legislatures. And in presidential elections the nature of electors is kind of strange about this.

andrewla commented on This website is for humans   localghost.dev/blog/this-... · Posted by u/charles_f
ggoo · 11 days ago
Particle board:

- degrades faster, necessitating replacement

- makes the average quality of all wood furniture notably worse

- arguably made the cost of real wood furniture more expensive, since fewer people can make a living off it.

Not to say the tradeoffs are or are not worth it, but "80% of the real thing" does not exist in a vacuum, it kinda lowers the quality on the whole imo.

andrewla · 11 days ago
> it kinda lowers the quality

That's why it's "80% of the real thing" and not "100% of the real thing".

andrewla commented on Can You Gerrymander Your Party to Power?   nytimes.com/interactive/2... · Posted by u/jaredwiener
andrewla · 11 days ago
The whole discourse around gerrymandering is fundamentally broken because national politics is too influential compared to local politics.

What should happen is that candidates should position themselves based on the districts that they represent. A West Virginia Democrat is a different creature than a New York Democrat, and a New York Republican is different than a West Virginia Republican. This is how it should be! The candidates should be trying to win races in their districts.

Instead we have this horrifying attempt to fix the parties in time and space and say "this is what the national party represents" and try to shape districts to align to those parties. This is the tail wagging the dog.

In a perfect world the districts should be shaped based on common interests, not based on voting record. Geography or population density are pretty good heuristics. Voting record is not because they are votes for candidates, not for parties.

andrewla commented on Can modern LLMs count the number of b's in "blueberry"?   minimaxir.com/2025/08/llm... · Posted by u/minimaxir
bachittle · 12 days ago
OpenAI definitely tarnished the name of GPT-5 by allowing these issues to occur. It's clearly a smaller model optimized for cost and speed. Compare it to GPT-4.5 which didn't have these errors but was "too expensive for them".

This is why Anthropic naming system of haiku sonnet and opus to represent size is really nice. It prevents this confusion.

andrewla · 12 days ago
> OpenAI definitely tarnished the name of GPT-5 by allowing these issues to occur

For a certain class of customer maybe that is true.

But the reality is that the fact that this occurs is very encouraging -- they are not micro-optimizing to solve cosmetic problems that serve no functional purpose. They are instead letting these phenomena serve as external benchmarks of a sort to evaluate how well the LLM can work on tasks that are outside of its training data, and outside of what one would expect the capabilities to be.

andrewla commented on Can modern LLMs count the number of b's in "blueberry"?   minimaxir.com/2025/08/llm... · Posted by u/minimaxir
Kwpolska · 12 days ago
The Internet has been poking fun at LLMs failing to count letters for many months. Are the AI companies really living in an echo chamber? They should have implemented a thing to count letters (and to search for palindromes, and…), and just have the LLM pass the request to the count_letters function, as they do for many other tasks already…
andrewla · 12 days ago
Why on earth would they do this? This is not a fundamentally useful task; it serves as a measure of the LLM's ability to generalize to tasks outside of its training data and that strain the limits of what it can express.

u/andrewla

KarmaCake day9054March 22, 2011View Original