I'm not sure what mathematical principle you could invoke that would make sense?

Roger Penrose made an argument like we can know Gödel's theorem is true without being able to prove it but AI can't, but I think you can figure both are guessing in a similar pattern recognising kind of way.

> I think [AGI is] alchemy-level nonsense.

Fair enough but he doesn't give much factual reasoning to support that. If you believe the brain is a biological computer and AI computing keeps advancing, at some point it will be able to do the same stuff or better, which is what most people think of as AGI.

I wrote about that for my uni entrance exam 43 years ago and it's always just seemed obvious common sense to me. I know Turing wrote about it before then but I never read that - it's just seems kind of obvious it'll happen.

Yes sorry that was implied - I personally wouldn't describe LLMs as capable of generating essays because what they produce is sub-par and mostly factual (as opposed to reliable), so I don't find their output useful except as a prompt or starting point for a human to then edit (similar to much of their other work).

I have made some minor games in JS with my kids with one for example, and managed to get it to produce a game of asteroids and pong with them (probably heavily based on tutorials scraped from the web of course). I had less success trying to build frogger (again probably because there are not so many complete examples). Anything truly creative/new they really struggle with, and it becomes apparent they are pattern matching machines without true understanding.

I wouldn't describe LLMs as useful at present and do not consider them intelligent in any sense, but they are certainly interesting.

> It literally is "in our training data similar concepts were clustered with some other similar concepts, and manipulating these clusters lead to different outcomes".

Recognizing concepts, grouping and manipulating similar concepts together, is what “abstraction” is. It's the fundamental essence of both "building a world model" and "thinking".

> Nothing in the link you provided is even close to "neurons, model of the world, thinking" etc.

I really have no idea how to address your argument. It’s like you’re saying,

“Nothing you have provided is even close to a model of the world or thinking. Instead, the LLM is merely building a very basic model of the world and performing very basic reasoning”.

This made me chuckle. But at the same time a bit uneasy because this could mean my own way of expressing myself online might've been impacted by reading too much AI drivel whilst trying to find signal in the noisy internet landscape...

The off-topic mentioning of graphics programming was because I tend to type as I think, then make corrections, and as I re-read it now the paragraph isn't great. The intent was to give an example of how I keep myself sharp, and challenging what many consider "settled" knowledge, where graphics programming happens to be my most recent example.

For what it's worth, English isn't my native language, and you'll have to take my word for it that no chat bots were used in generating any of the responses I've made in this thread. The fact that people are already uncertain about who's a human and who's not is worrisome.