Now, success in a tough math exam isn't "automating all human labor" but it is certainly a benchmark many thought AI would not achieve easily. Even so, many are claiming it isn't really a big deal, and that humans will still be far smarter than AI's for the foreseeable future.
My question is, if you are in the aforementioned camp, what would it take you to adopt a frame of mind roughly analogous to "It is realistic that AI systems will become smarter than humans, and could automate all human labor and cognitive outputs within a single-digit number of years".
Would it require seeing a humanoid robot perform some difficult task? (the Metaculus definition of AGI requires that a robot be able to satisfactorily assemble a (or the equivalent of a) circa-2021 Ferrari 312 T4 1:8 scale automobile model.). Would it involve a Turing test of sufficient rigor? I'm curious what people's personal definition of "ok this is really real" is.
We aren't getting that with next-token generators. I don't think we'll get there by throwing shit at the wall and seeing what sticks, either, I think we'll need a deeper understanding of the mind and what intelligence actually is before we can implement it on our own, virtually or otherwise.
Similarly, we're pretty good at creating purpose-built machines, but when it comes to general/universal purpose, it's still in its infancy. The hand is still the most useful universal tool we have. It's hard to compete with the human mind + body when it comes to adapting to and manipulating the environment with purpose. There's quite literally billions of them, they create themselves and their labor is really cheap, too.
There's my serious answer.
This isn't an exhaustive list, it's just intended to illustrate when I'd start seriously thinking AGI was possible with incremental improvements.
I take AI seriously in the sense that it's useful, can solve problems, and represents a lot of value for a lot of businesses, but not in the sense that I think the current methods are headed for AGI without further breakthroughs. I'm also not an expert.
From OP's other comments:
> A lot of people here have an emotional aversion to accepting AI progress. They’re deep in the bargaining/anger/denial phase.
I too wish AI progress wasn't happening as fast as it is. I, as a software developer, want to imagine a future where my skills are useful. However I haven't seen much convincing evidence or arguments on this site that appropriately critique short-term AI timelines that don't resort to logical fallacies, name calling, ad-hominems, or other tired attempts at zingers that contribute nothing to the discourse.
Defined as what, by who?
> a constrained problem that someone made up
How is that different from what humans do when asking questions?
Dead Comment
And let’s say AI does automate all human labor… what’s the plan? That happening, will lead to chaos, without some massive changes in how society is organized and functions. That change itself with be chaotic and massively disruptive and painful for millions. If someone can’t answer that question, they have no business hyping up the end of human involvement in civilization.
Capitalism is driving this hype around cost cutting with AI, but capitalism requires people have capital to buy various goods and services. Where is that going to come from when unemployment hits 100%? Who are the customers?
Why would anyone be excited about this future before solving for this problem?
"In the 60s, we wanted to build computers that acted like people. Not just people, but smart people. So what do smart people do? We play chess! So we spent a lot of time beating chess and learned basically nothing about AI."
Beating the Math Olympiad strikes me as much the same. They're solving "hard" problems, but not solving easy ones.
I want a robot that can clean a toilet. Hand it a brush, send it into the room, and get it clean. Then have it make the bed, without crushing any of the stuff strewn haphazardly about. Something humans do for minimum wage because anybody at all can do it.
Physical manipulation of the real world isn't strictly required for AI, but as a test it precludes a lot of solving the hard problem without solving the 'easy' problem. The real world is very unforgiving of automata in uncontrolled circumstances, something that animals (not just humans) do with minimal effort.
Because then I know: they have been using it for real human benefit without trying to get humans hooked on re-occurring costs.
When it starts solving actual human problems like climate or start filling in our gaps of knowledge in science. When it starts lifting humans up to higher grounds instead of replacing them to make a buck.
Why do I even need to make up my mind about it?