It always surprises me at the ease at which people jump on a) imminent AGI and b) human extinction in the face of AGI. Would love for someone to correct me / add information here to the contrary. Generalist here just refers to a "multi-faceted agent" vs "General" like AGI.
For a) - I see 2 main blockers,
1) A way to build second/third order reasoning systems that rely on intuitions that haven't already been fed into the training sets. The sheer amount of inputs a human baby sees and processes and knows how to apply at the right time is an unsolved problem. We don't have any ways to do this.
2) Deterministic reasoning towards outcomes. Most statistical models rely on "predicting" outputs, but I've seen very little work where the "end state" is coded into a model. Eg: a chatbot knowing that the right answer is "ordering a part from amazon" and guiding users towards it, and knowing how well its progressing to generate relevant outputs.
For (b) -- I doubt human extinction happens in any way that we can predict or guard against.
In my mind, it happens when autonomous systems optimizing reward functions to "stay alive" (by ordering fuel, making payments, investments etc) fail because of problems described above in (a) -- the inability to have deterministic rules baked into them to avoid global fail states in order to achieve local success states. (Eg, autonomous power plant increases output to solve for energy needs -> autonomous dam messes up something structural -> cascade effect into large swathes of arable land and homes destroyed).
Edit: These rules can't possibly all be encoded by humans - they have to be learned through evaluation of the world. And we have not only no way to parse this data at a global scale, but also develop systems that can stick to a guardrail.
I am quite scared of human extinction in the face of AGI. I certainly didn't jump on it, though! I was gradually convinced by the arguments that Yudkowsky makes in "Rationality: from AI to Zombies" (https://www.readthesequences.com/). Unfortunately they don't fit easily into an internet comment. Some of the points that stood out to me, though:
- We are social animals, and take for granted that, all else being equal, it's better to be good to other creatures than bad to them, and to be truthful rather than lie, and such. However, if you select values uniformly at random from value space, "being nice" and "being truthful" are oddly specific. There's nothing universally special about deeply valuing human lives any more so than say deeply valuing regular heptagons. Our social instincts are very ingrained, though, making us systematically underestimate just how little a smart AI is likely to care whatsoever about our existence, except as a potential obstacle to its goals.
So here's hoping you're right about (a). The harder AGI is, the longer we have to figure out AI alignment by trial and error, before we get something that's truly dangerous or that learns deception.
The human extinction due to would be "hard takeoff" of an AGI should be understood as a thought experiment, conceived in a specific age when the current connectionist paradigm wasn't yet mainstream. The AI crisis was expected to come from some kind of "hard universal algorithmic artificial intelligence", for example AIXItl undergoing a very specific process of runaway self-optimization.
Current-generation systems aka large connectionist models trained via gradient descent simply don't work like that: they are large, heavy, continuous, the optimization process giving rise to them does so in smooth iterative manner. Before hypothetical "evil AI" there will be thousands of iterations of "goofy and obviously erroneously evil AI", with enough time to take some action. And even then, current systems including this one are more often than not trained with predictive objective, which is very different compared to usually postulated reinforcement learning objective. Systems trained with prediction objective shouldn't be prone to becoming agents, much less dangerous ones.
If you read Scott's blog, you should remember the prior post where he himself pointed that out.
In my honest opinion, unaccountable AGI owners pose multiple OOM more risk than alignment failure of a hypothetical AI trying to predict next token.
We should think more about the Human alignment problem.
We're going to have a harder problem with AI that thinks of itself as human and expects human rights than we are with AI that thinks of humans as 'other' and disposable.
We're making it in our image. Literally.
Human social good isn't some inherent thing to the biology of the brain. There are aspects like mirror neurons and oxytocin that aid its development, but various "raised by wolves" case studies have shown how damaging not having exposure to socialization information during developmental periods of neuroplasticity is on humans and later integration into society.
We're building what's effectively pure neuroplasticity and feeding it almost all the data on humanity we can gather as quickly as we can.
What comes out of that is going to be much more human than a human child raised by dogs or put in an isolation box.
Don't get so caught up in the body as what makes us quintessentially human. It's really not.
I think human extinction through human stupidity or hubris is much much much more likely than through an unpredictable path down general AI.
For example, some total whack job of an authoritarian leader is in charge of a sufficient nuclear arsenal and decides to intimidate an adversary by destroying a couple minor cities, and the situation escalates badly. (stupidity)
Or we finally pollute our air and/or water with a persistent substance that either greatly reduces human life span or reproduction rate. (hubris)
I think either of the above is more likely to occur, and I am not commenting on current world events in any way. I think when something bad finally happens, it is going to come completely out of left field. Dr Strangelove style.
And the last of us will be saying "Hmmm, I didn't see that coming".
I think of it as System 1 vs System 2 thinking from 'Thinking, Fast and Slow' by Daniel Kahneman.[1]
Deep learning is very good at things we can do without thinking, and is in some cases superhuman in those tasks because it can train on so much more data. If you look at the list of tasks in System 1 vs System 2, SOTA Deep learning can do almost everything in System 1 at human or superhuman levels, but not as many in System 2 (although some tasks in System 2 are somewhat ill-defined), System 2 builds on system 1. Sometimes superhuman abilities in System 1 will seem like System 2. (A chess master can beat a noob without thinking while the noob might be thinking really hard. Also GPT-3 probably knows 2+2=4 from training data but not 17 * 24, although maybe with more training data it would be able to do math with more digits 'without thinking' ).
System 1 is basically solved, but System 2 is not. System 2 could be close behind System 2 by building on System 1 but it isn't clear how long that will take.
In biological history, system two was an afterthought at best. It likely didn't exist before spoken language, and possibly barely before written language. And to the extent that system two exists, it's running on hardware almost entirely optimized for system one thinking.
It remains to be asked, just why this causal, counterfactual, logical reasoning cannot emerge in a sufficiently scaled-up model trained on a sufficiently diverse real world data?
Neural networks, at the end of the day, are still advanced forms of data compression. Since they are Turing-complete it is true that given enough data they can learn anything, but only if there is data for it. We haven't solved the problem of reasoning without data, i.e. without learning. The neural network can't, given some new problem that has never appeared in the dataset, in a deterministic way, solve that problem (even given pretrained weights and whatnot). I do think we're pretty close but we haven't come up with the right way of framing the question and combining the tools we have. But I do think the tools are there (optimizing over the space of programs is possible, learning a symbol-space is possible, however symbolic representation is not rigorous or applicable right now)
Good point. This gets us into the territory of not just "explainable" models, but also the ability to feed into those models "states" in a deterministic way. This is a merger of statistical and symbolic methods in my mind -- and no way for us to achieve this today.
> it happens when autonomous systems optimizing reward functions fail because of problems described above in (a) -- the inability to have deterministic rules baked into them to avoid global fail states in order to achieve local success states.
yes, and there is an insight here that I think is often missed in the popular framing of AI x-risk: the autonomous systems we have today (which, defined broadly, need not be entirely or even mostly digital) are just as vulnerable to this
the AGI likely to pose extinction risk in the near term has humans in the loop
less likely to look like Clippy, more likely to look like a catastrophic absence of alignment between loci of agency (social, legal, technical, corporate, political, etc)
>In my mind, it happens when autonomous systems optimizing reward functions to "stay alive" (by ordering fuel, making payments, investments etc) fail because of problems described above in (a) -- the inability to have deterministic rules baked into them to avoid global fail states in order to achieve local success states. (Eg, autonomous power plant increases output to solve for energy needs -> autonomous dam messes up something structural -> cascade effect into large swathes of arable land and homes destroyed).
And for this to develop in machines, machines would have to be subject to many mistakes along the way leading to all kinds of outcomes that we hold humans accountable for by fining them, sending them to jail, some of them dying etc. I think that would be so wholly unpalatable to man kind they'd cut that experiment short before it ever reached any sort of scale.
I agree with your conclusion that enough of the rules can't be encoded by us as we don't even know them and for machines to acquire them the traditional way is, I believe, fundamentally disagreeable to humans.
> A way to build second/third order reasoning systems...
I've been pondering about this problem for a while now[1], Could we build a collective intelligence through community submitted recipes for second-order decisions for various common activities via generalized schema?
I didn't think about addressing this for AI, But as an aid for us in multi-order thinking. But now that you mention it to be a barrier for AGI, It does make sense.
The concern over AI safety seems about right. The unique thing is that anybody cares at all about civilisational negative externalities in a functional and reasonable way. This is quite rare, unprecedented even. Typically humanity just rushes into new endeavours with little concern and leaves future generations to pick up the pieces afterwards (social media, colonialism, processed food, nuclear weapons etc).
>2) Deterministic reasoning towards outcomes. Most statistical models rely on "predicting" outputs, but I've seen very little work where the "end state" is coded into a model. Eg: a chatbot knowing that the right answer is "ordering a part from amazon" and guiding users towards it, and knowing how well its progressing to generate relevant outputs.
Here's a real generalist question? What's the point of conversation? In the sense that its a "social game", what are the strategy sets on a given turn, and what does it even mean to "win"? Forget about Artificial Bullshiters vs Artificial Idea-what-to-say. How can we even speculate if ABS or "real AI" solves the problem better when we don't really specify the problem space or how to recognize a solution?
In terms of calls and appropriate responses and responses to responses and terminal states, what even is conversation at the protocol level?
Personally I'm more worried about what effects sophisticated "dumb" AI will have on human culture and expectations. It would have been hard to imagine we'd be living in this world with our attention assailed by platforms available anywhere when Jobs first held up the iPhone to the world. Just the same, I am not sure that the impending cultural effects from advancements in AI are fully understood at this moment.
For example, I wonder what expectations we will set for ourselves when we pick up a pencil once it becomes common knowledge that production-ready digital art can be created by an AI within minutes. What will people be saying about "old art" in those days? What will we think of people that deliberately choose to disavow AI art?
> worried about what effects sophisticated "dumb" AI will have on human culture and expectations
It is already happening, but it is not a new phenomenon - just the prosecution of the effects of widespread inadequate education.
The undesirable effect namely is, an increase of overvaluing cheap, thin products - a decrease in recognizing actual genius, "substance". For example, some seem to be increasingly contented with "frequentism" as if convinced that language is based on it - while it is of course the opposite - one is supposed to state what is seen, and to state "plausible associations" would have been regarded as a clear sign of malfunction. There seems to be an encouragement to take for "normal" what is in fact deficient.
Some people are brought to believe that some shallow instinct is already the goal and do not know "deep instinct", trained on judgement (active digestion, not passive intake).
The example provided fits:
> production-ready digital art
For something to be artistic it has to have foundational justifications - a mockery of art is not art. The Artist chose to draw that line under a number of evaluations that made it a good choice, but an imitator, even in a copy, used the ("frequentist", by the way note) evaluation that the other is an Artist - and there is a world of depth difference between the two.
The difference is trivial, and yet, many are already brought to confuse the shallow mockery and the deep creation.
For a).1). you basically argued in favor of scalability. We don't know if we are on a training set more or less than a typical human at this point. If I would guess, I think high-bandwidth networked computers can be much more efficient at gathering training set than a single human.
For a).2). you argued in favor of symbolic reasoning. My personal interpretation of that line of thinking is: it helps for us to understand complex thinking machines, but it is not necessarily the building block for the thinking machine.
In the area of AI, there are a lot of opinions, but at the end of the day, builders win the argument. So far, what builders have shown gives me optimistic hope.
For those worried about a threats for AGI, this is why such a system must be fully explainable. It's like thinking about a database with no validation at all, if you enter the wrong command you could destroy the integrity of the data. If you have constraints in place you're safer, and of course with SQL you can explain the data and inconsistencies.
My own effort, which focusing on natural language understanding: https://lxagi.com
For me at least, the fear is not so much about the specifics, but more around the fact of what exponential curves look like. At any point, everything before looks basically horizontal and anything after looks vertical. In that sense, the fear is that while things seem quite behind right now, it could in an instant zoom past us before we even have the time to realize it. It is partly rooted in science fiction.
I think the "AGI wants to kill us" meme is just to get us ready for the moment when the authorities unleash the killer robots on us because there's too many of us on the planet. "Whoops, those robots did it all by themselves because it's just inevitable that AGI was going to do that. Haven't you watched a sci-fi movie in the last 50 years?"
I'm not so sure its impossible. the 40 year semantic map project Douglas Lenat: Cyc is truly astounding. I think in the next decade we will see really interesting integration of state of the art deep learning with something like Cyc
I've always assumed that the trouble will begin when we have a model for 'true' AGI and discover that the constraint "do not harm any human" renders it functionally inert.
The paradoxical idea that AGI is going to still be a monkey paw following simplistic paradigms of operational goals in disastrous ways is hilarious to me every time I see it.
I increasingly wonder at what point we'll realize that humans aren't actually particularly good at what we've specialized into (there's just not much competition), and our failure to picture what 'better' looks like may be less about the impossibility of better to exist than it is the impossibility of it for humans to picture it.
I keep seeing predictions of what will be impossible for AI because it treads on our perceived human uniqueness (i.e. sure AI beat a human at chess but it'll be 25+ years before it will beat us at Go) needing to get walked back, and yet we continue to put forward a new iteration of that argument at every turn.
Maybe AI will turn out to be better at identifying what's good for humanity than humanity is. Because frankly, humanity is downright awful at that skill and has been for pretty much its entire existence.
(Former emerging tech consultant for ~10% of Fortune 500 here)
(a) I've noticed a common trend of AI researchers looking at the tree in front of them and saying "well, this tree is not also a forest and won't be any time soon."
But there's not always awareness of what's going on in other specialized domains, so an AI vision researcher might not be intimately aware of what's currently being done in text or in "machine scientists" in biology for example.
As well, it overlooks the development of specialization of the human brain. We have some specialized structures that figured their niche out back with lizards, and others that developed much later on. And each of those specialized functions work together to give rise to 'human' intelligence.
So GPT-3 might be the equivalent of something like the Wernicke's area, and yes - on its own it's just a specialized tool. But what happens as these specialized tools start interconnecting?
Throw GPT-3 together with Dall-E 2 and the set of use cases is greater than just the sum of the parts.
This is going to continue to occur as long as specialized systems continue to improve and emerge.
And quickly we'll be moving into territory where orchestration of those connections is a niche that we'll both have data on (from human usage/selection of the specialist parts) and will in turn build meta-models to automate sub-specialized models from that data.
Deterministic reasoning seems like a niche where a GAN approach will still find a place. As long as we have a way for one specialized model to identify "are these steps leading to X" we can have other models only concerned with "generate steps predicted to lead to X."
I don't think we'll see a single model that does it all, because there's absolutely no generalized intelligence in nature that isn't built upon specialized parts anyways, and I'd be surprised if nature optimized excessively inefficiently in that progress.
Will this truly be AGI in a self-determining way? Well, it will at least get closer and closer to it with each iteration, and because of the nature of interconnected solutions, will probably have a compounding rate of growth.
In a theoretical "consciousness" sense of AGI, I think the integrated information theory is interesting, and there was a paper a few years ago about how there's not enough self-interaction of information possible in classical computing to give rise to consciousness, but we'll probably have photonics in commercial grade AI setups within two years, so as hand-wavy as the IIT theory is, the medium will be shifting towards one compatible with their view of consciousness-capable infrastructure much sooner for AI than quantum competing in general.
So I'd guess we may see AI that we're effectively unable to determine if it is "generally intelligent" or 'alive' within 10-25 years, though I will acknowledge that AI is the rare emerging tech that I've been consistently wrong about the timing on in a conservative direction (it keeps hitting benchmark improvements faster than I think it will).
(b) The notion AGI will have it out for us is one of the dumbest stances and my personal pet peeves out there, arguably ranked along with the hubris of "a computer will never be able to capture the je ne sais quoi of humanity."
The hands down largest market segment for AI is going to be personalization, from outsourcing our work to a digital twin of ourselves to content curation specific to our own interests and past interactions.
Within a decade, no one is going to give the slightest bit of a crap about interactions with other humans in a Metaverse over interacting with AIs convincingly human enough but with the key difference of actually listening to our BS rather than just waiting for their turn to talk.
There's a decent chance we're even going to see a sizable market for feeding social media data of deceased loved ones and pets into AI to make twins available in such settings (and Microsoft already holds a patent on that).
So do we really think humans are so repugnant that the AI which will eventually reach general intelligence within the context of replicating itself as ourselves, as our closet friends and confidants, as our deceased loved ones - will suddenly decide to wipe us out? And for what gains? What is AI going to selfishly care about land ownership and utilization for?
No. Even if some evolved AGI somehow has access to DARPA killer drones and Musk's Terminator robots and Boston Dynamics' creepy dogs, I would suspect a much likelier target would be specific individuals responsible for mass human suffering the AIs will be exposed to (pedophiles, drug kingpins, tyrants) than it is grandma and little Timmy.
We're designing AI to mirror us. The same way some of the current thinking of how empathy arises in humans is from our mirror neurons and the ability to put ourselves in the shoes of another, I'm deeply skeptical of the notion that AI which we are going to be intimately having step into human shoes will become some alien psychopath.
It’s a very entertaining read from a couple of years ago (I think I’ve read it in 2017), and man, have things happened in the field since then. If feels like things truly start coming together. Transformers and then some incremental progress look like a very, very promising avenue. I deeply wonder in which areas this will shape the future more than we are able to anticipate beforehand.
Not you specifically, but I honestly don't understand how positive many in this community (or really anyone at all) can be about these news. Tim Urban's article explicitly touches on the risk of human extinction, not to mention all the smaller-scale risks from weaponized AI. Have we made any progress on preventing this? Or is HN mostly happy with deprecating humanity because our replacement has more teraflops?
Even the best-case scenario that some are describing, of uploading ourselves into some kind of post-singularity supercomputer in the hopes of being conscious there, doesn't seem very far from plain extinction.
I think the best-case scenario is that 'we' become something different than we are right now. The natural tendency of life(on the local scale) is toward greater information density. Chemical reactions beget self-replicating molecules beget simple organisms beget complex organisims beget social groups beget tribes beget city states beget nations beget world communities. Each once of these transitions looks like the death of the previous thing and in actuality the previous thing is still there, just as part of a new whole. I suspect we will start with natural people and transition to some combination of people whose consciousness exists, at least partially, outside of the boundaries of their skulls, people who are mostly information on computing substrate outside of a human body, and 'people' who no longer have much connection with the original term.
And that's OK. We are one step toward the universe understanding itself, but we certainly aren't the final step.
I feel exactly the opposite. AI has not yet posed any significant threats to humanity other than issues with the way people choose to use it (tracking citizens, violating privacy, etc.).
So far, we have task-driven AI/ML. It solves a problem you tell it to solve. Then you, as the engineer, need to make sure it solves the problem correctly enough for you. So it really still seems like it would be a human failing if something went wrong.
So I'm wondering why there is so much concern that AI is going to destroy humanity. Is the theoretical AI that's going to do this even going to have the actuators to do so?
Philosophically, I don't have an issue with the debate, but the "AI will destroy the world" side doesn't seem to have any tangible evidence. It seems to me that people seem to take it as a given that it's possible AI could eliminate all of humanity and they do not support that argument in the least. From my perspective, it appears to be fearmongering because people watched and believed Terminator. It appears uniquely out-of-touch.
Agreed. People think of the best case scenario without seriously considering everything that can go wrong. If we stay on this path the most likely outcome is human extinction. Full stop
> Or is HN mostly happy with deprecating humanity because our replacement has more teraflops?
If we manage to make a 'better' replacement for ourselves, is it actually a bad thing? Our cousin's on the hominoid family tree are all extinct, yet we don't consider that a mistake. AI made by us could well make us extinct. Is that a bad thing?
We should ask our compute overlords to perform their experiments in as open environment as possible, just because we, the public, should have the power to oversee the exact direction this AI revolution is taking us.
If you think about it, AI safetyism is a red herring compared to a very real scenario of powerful AGIs working safely as intended, just not in our common interest.
The safety of AGI owners' mindset seems like a more pressing concern compared to a hypothetical unsafety of a pile of tensors knit together via gradient descent over internet pictures.
That Tim Urban piece is great. It's also an interesting time capsule in terms of which AI problems were and were not considered hard in 2015 (when the post was written). From the post:
> Build a computer that can multiply two ten-digit numbers in a split second—incredibly easy. Build one that can look at a dog and answer whether it’s a dog or a cat—spectacularly difficult. Make AI that can beat any human in chess? Done. Make one that can read a paragraph from a six-year-old’s picture book and not just recognize the words but understand the meaning of them? Google is currently spending billions of dollars trying to do it. Hard things—like calculus, financial market strategy, and language translation—are mind-numbingly easy for a computer, while easy things—like vision, motion, movement, and perception—are insanely hard for it.
The children's picture book problem is solved; those billions of dollars were well-spent after all. (See, e.g., DeepMind's recent Flamingo model [1].) We can do whatever we want in vision, more or less [2]. Motion and movement might be the least developed area, but it's still made major progress; we have robotic parkour [3] and physical Rubik's cube solvers [4], and we can tell a robot to follow simple domestic instructions [5]. And Perceiver (again from DeepMind [6]) took a big chunk out of the perception problem.
Getting a computer to carry on a conversation [7], let alone draw art on par with human professionals [8], weren't even mentioned as examples, so laughably out of reach they seemed in the heathen dark ages of... 2015.
And as for recognizing a cat or a dog — that's a problem so trivial today that it isn't even worth using as the very first example in an introductory AI course. [9]
If someone re-wrote this post today, I wonder what sorts of things would go into the "hard for a computer" bucket? And how many of those would be left standing in 2029?
> And as for recognizing a cat or a dog — that's a problem so trivial today
Last time I checked - though it's been a long while I could not check thoroughly owing to other commitments - "«recognizing»" there was "consistently successfully guessing", not "critically defining". It may be that the problem was solved in the latest years, I cannot exclude it - but I have not seen around in the brief "news checking" exercise the signals required for the solution.
That's a poor usage of "just": discovering that "X is just Y" doesn't diminish X; it tells us that Y is a much more complex and amazing topic than we might have previously thought.
For example: "Life is just chemistry", "Earth is just a pile of atoms", "Behaviours are just Turing Machines", etc.
It’s most fascinating (or very obvious) - look at Conway’s Game of Life, then scale it up - a lot. Unlimited complexity can arise from very simple rules and initial conditions.
Now consciousness on the other hand is unfathomable and (in its finitude) extremely depressing for me.
Stop being depressed because it simply, clearly, certainly, is not. I just wrote a few paragraphs about it in an immediately previous post. This confirms that this phase is getting some people fooled on basics.
(I really like the framing of "weakly general AI" since it puts the emphasis on the generality and not whether it's a superintelligence)
Edit: Probably not today, but mostly because 1.2B parameters isn't enough to get it the high winograd scores that PaLM etc have. But it seems pretty clear you could scale this architecture up and it will likely pass. The question is when someone will actually train a model that can do it
I think this is a step in the right direction, but the performance on most tasks is only mediocre. The conversation and image captioning examples in the paper are pretty bad, and even on some relatively simple control tasks it performs surprisingly poorly.
That's not to say it's not an important step. Showing that you can train one model on all of these disparate tasks at once and not have the system completely collapse is a big deal. And it lays the foundation for future efforts to raise the performance from "not totally embarrassing" to "human level". But there's still a ways to go on that front.
Agreed, I think if they were to drop the real-time constraint for the sake of the robotics tasks, they could train a huge model with the lessons from PaLM and Chincilla and probably slam dunk the weakly general AI benchmark.
Before you visualize a straight path between "a bag of cool ML tricks" and "general AI", try to imagine superintelligence but without consciousness. You might then realize that there is no obvious mechanism which requires the two to appear or evolve together.
It's a curious concept, well illustrated in the novel Blindsight by Peter Watts. I won't spoil anything here but I'll highly recommend the book.
>"try to imagine superintelligence but without consciousness."
The only thing that comes to mind is how many different things come to mind to people when the term "superintelligence" is used.
The thing about this imagination process, however, is that what people produce is a "bag of capacities" without a clear means to implement those capacities. Those capacities would be "beyond human" but in what direction probably depends on the last movie someone watched or something similarly arbitrary 'cause it certainly doesn't depend on their knowledge of a machine that could be "superintelligent", 'cause none of us have such knowledge (even if this machine could go to "superintelligence", even our deepmind researchers don't know the path now 'cause these are being constructed as a huge collection of heuristics and what happens "under the hood" is mysterious to even the drivers here).
Notably, a lot of imagined "superintelligences" can supposedly predict or control X, Y or Z thing in reality. The problem with such hypotheticals is that various things may not be much more easily predictable by an "intelligence" than by us simply because such prediction involves imperfect information.
And that's not even touch how many things go by the name "consciousness".
Likely insufficient but here is a shot at a materialist answer.
Consciousness is defined as an entity that has an ethical framework that is subordinated to it's own physical existence, maintaining that existence, and interfacing with other conscious entities as if they also have an ethical framework with similar parameters who are fundamentally no more or less important/capable than itself.
Contrast with non-conscious super-intelligence that lacks physical body (likely distributed). Without a physical/atomic body and sense data it lacks the capacity to empathize/sympathize as conscious entities (that exist within an ethical framework that is subordinated to those limitations/senses) must. It lacks the perspective of a singular, subjective being and must extrapolate our moral/ethical considerations, rather than have them ingrained as key to it's own survival.
Now that I think about it, it's probably not much different than the relationship between a human and God, except that in this case it's: a machine consciousness and a machine god.
To me, the main problem is that humans (at large) have yet to establish/apply a consistent philosophy with which to understand our own moral, ethical, and physical limitations. For the lack of that, I question whether we're capable of programming a machine consciousness (much less a machine god) with a sufficient amount of ethical/moral understanding - since we lack it ourselves (in the aggregate). We can hardly agree on basic premises, or whether humanity itself is even worth having. How can we expect a machine that we make to do what we can't do ourselves? You might say "that's the whole point of making the machine, to do something we can't" but I would argue we have to understand the problem domain first (given we are to program the machine) before we can expect our creations to apply it properly or expand on it in any meaningful way.
That's exactly what Peter Watts spends 200 pages discussing, in between first contact, cognitive malfunctions, telematter drives, resurrected vampire paleogenetics and a very healthy dose of unreliable-narration.
Not sure what you mean by consciosness here, but one definition of intelligence I've seen was "it's what establishes relationship between things" and the very first relationship it establishes is "me vs not-me".
Intelligence is the system allowing the redefinition of ideas over the entities constituting an inner representation of a world: how a non-trivial system of "consciousness" (otherwise the use of the term would be a waste) could have to be employed for that?
1. tell me about a cat (given a prompt such as "describe a cat to me")
2. recognize a cat in a photo, and describe the cat in the photo
does the model understand that a cat that it sees in an image is related to a cat that it can describe in natural language?
As in, are these two tasks (captioning an image and replying to a natural language prompt) so distinct that a "cat" in an image excites different neurons than a "cat" that I ask it about? Or is there overlap? Or we don't know :)
I wonder if you could mix the type of request. Like, provide a prompt that is both text and image. Such as "Here is a picture of a cat. Explain what breed of cat it is and why you think so." Possibly this is too advanced for the model but the idea makes me excited.
Definitely possible. OpenAI's CLIP model already embeds images and text into the same embedding space.
I don't know exactly how this particular model works but it is creating cross modal relationships otherwise it would not have the capacity to be good at so many tasks.
How confident are we that it doesn't just have basically 600 smaller models and a classifier telling it which to use? Seems like it's a very small model (by comparison), which is certainly a mark in it's favor.
> OpenAI's CLIP model already embeds images and text into the same embedding space.
Not really. Embeddings from images occupy a different region of the space than embeddings from text. A picture of cat and the text "cat" do not resolve to the same embedding.
This is why DALL-E has a model learning to translate CLIP text embeddings into CLIP image embeddings before decoding the embedding.
I think the critical question here is does it have a concept of cattyness? This to me is the crux of a AGI: can it generalise concepts across domains?
Moreover, can it relate non-cat but cat-like objects to it's concept of cattyness? As in, this is like a cat because it has whiskers and pointy ears, but is not like a cat because all cats I know about are bigger than 10cm long. It also doesn't have much in the way of mouseyness: it's aspect ratio seems wrong.
I don't disagree with you, and I think that what you're saying is critical; but it feels more and more like we are shifting the goalposts. 5 years ago; recognizing a cat and describing a cat in an image would be incredible impressive. Now, the demands we are making and the expectations we keep pushing feel like they are growing as if we are running away from accepting that this might actually be the start of AGI.
> As in, are these two tasks (captioning an image and replying to a natural language prompt) so distinct that a "cat" in an image excites different neurons than a "cat" that I ask it about? Or is there overlap? Or we don't know :)
We only have very limited (but suggestive) evidence that the human brain has abstract "cat" neurons involved in various sensory and cognitive modes. Last time I paid attention, there was reasonably strong evidence that an image of a cat and reading the word cat used some of the same neurons. Beyond that it was pretty vague, there seemed to be evidence of a network that only weakly associates concepts from different modes, which isn't consistent with most people's subjective experience.
But then we have other evidence that what we think of as our subjective cognitive experience is at least partly a post-hoc illusion imposed for no apparent reason except that it creates an internal narrative consistency (which presumably has some utility, possibly in terms of having a mind others can form theories about more readily).
"The same network with the same weights can play Atari, caption images, chat, stack blocks with a real robot arm and much more, deciding based on its context whether to output text, joint torques, button presses, or other tokens."
This is rather mind blowing. Does it also mean that the generalist network is smaller than the sum of all specialist networks that are equivalent? Even if not, I find the idea that a single network can be used for such diverse tasks at all highly fascinating.
I don't find it surprising that a single network can do all those things with appropriate formatting of the data. In itself it just means the network has a large enough capacity to learn all the different tasks.
The interesting questions imo, which they studied, is what kind of added generalization takes place by learning across the different tasks. For example, does learning multiple tasks make it better at a given task than a model that is just trained for one task, and can it generalize to new tasks (out of distribution).
They looked at how it performed on held out tasks (see fig 9 in the paper). I'm still getting my head around the result though so couldn't summarize their finding yet.
Yeah, Figure 9 is the money figure in this paper and it actually splashes some cold water on the claims in the rest of the paper. While it generalizes OK to some tasks that are held out, it does pretty poorly on the Atari boxing task, which they openly admit is quite different from the others. Gato seems more likely to be a competent attempt at brute forcing our way towards weak general AI, which is a valid approach, but the question then will always be how does it do with something its never seen before, and how do you possibly brute force every possible situation? I think we're heading more towards a constellation of very intelligent expert machines for particular tasks that may be wrapped into a single package, but that are not strong AI.
Many networks just predict the next integer in a sequence of integers. It sounds like this model identifies what category of problem a sequence of integers falls into and then makes an accurate prediction for that sequence, as you would expect given what it was trained on.
It's interesting I so often see the discussion of AI in the context of what a singular model can or can't do.
This seems myopic.
Our own brains have specialized areas. Someone who has a stroke impacting Broca's region isn't going to have a fun time trying to do what that region was specialized for.
And as a society, I don't think we'd have gotten very far if every individual was expected to perform every task.
Where generalization in AI seems like the holy grail is not in replacing more specialized models, but in orchestrating them.
As impressive as this effort at generalization is, unless the work here is going to translate into orchestration and management of other specialized models, I don't think this is going to have much market relevance unless the generalized approach can somehow (in that generalization) gain an advantage in outperforming specialized approaches.
But Intelligence is one of those, and when people start calling naïvely calling "intelligence" its opposite "imprint", there is a problem. There are even the seeds of a social problem.
> orchestrating ... specialized models
Intelligence (ontology development) is one such specialized model which I have seen more faithfully and consciously attempted in classic AI as opposed to contemporary. Orchestration is crucial for the peripherals that allow the input/output. The study of "generalization" is none the less crucial, because we are to investigate the development of a system that can intrinsically, by its own nature, be exported to the largest number of relatable skills. Even with the highest specialization, up to physical, there is a matter of organicity: a hand is "specialized", yet a hook is not a hand.
So how long until someone trains one of these models to complete tasks by interacting directly with network/unix sockets?
At the moment, it seems like the model needs to be trained with each modality of data in mind at the start, but a generalised "supermodality" that can deliver all the others would allow truly generalised learning if the model were still capable of making sense of the input.
You'd obviously still need to finetune on any new modalities, but you wouldn't need to start from scratch.
Readit News
It always surprises me at the ease at which people jump on a) imminent AGI and b) human extinction in the face of AGI. Would love for someone to correct me / add information here to the contrary. Generalist here just refers to a "multi-faceted agent" vs "General" like AGI.
For a) - I see 2 main blockers,
1) A way to build second/third order reasoning systems that rely on intuitions that haven't already been fed into the training sets. The sheer amount of inputs a human baby sees and processes and knows how to apply at the right time is an unsolved problem. We don't have any ways to do this.
2) Deterministic reasoning towards outcomes. Most statistical models rely on "predicting" outputs, but I've seen very little work where the "end state" is coded into a model. Eg: a chatbot knowing that the right answer is "ordering a part from amazon" and guiding users towards it, and knowing how well its progressing to generate relevant outputs.
For (b) -- I doubt human extinction happens in any way that we can predict or guard against.
In my mind, it happens when autonomous systems optimizing reward functions to "stay alive" (by ordering fuel, making payments, investments etc) fail because of problems described above in (a) -- the inability to have deterministic rules baked into them to avoid global fail states in order to achieve local success states. (Eg, autonomous power plant increases output to solve for energy needs -> autonomous dam messes up something structural -> cascade effect into large swathes of arable land and homes destroyed).
Edit: These rules can't possibly all be encoded by humans - they have to be learned through evaluation of the world. And we have not only no way to parse this data at a global scale, but also develop systems that can stick to a guardrail.
- We are social animals, and take for granted that, all else being equal, it's better to be good to other creatures than bad to them, and to be truthful rather than lie, and such. However, if you select values uniformly at random from value space, "being nice" and "being truthful" are oddly specific. There's nothing universally special about deeply valuing human lives any more so than say deeply valuing regular heptagons. Our social instincts are very ingrained, though, making us systematically underestimate just how little a smart AI is likely to care whatsoever about our existence, except as a potential obstacle to its goals.
- Inner alignment failure is a thing, and AFAIK we don't really have any way to deal with that. For those that don't know the phrase, here it is explained via a meme: https://astralcodexten.substack.com/p/deceptively-aligned-me...
So here's hoping you're right about (a). The harder AGI is, the longer we have to figure out AI alignment by trial and error, before we get something that's truly dangerous or that learns deception.
Current-generation systems aka large connectionist models trained via gradient descent simply don't work like that: they are large, heavy, continuous, the optimization process giving rise to them does so in smooth iterative manner. Before hypothetical "evil AI" there will be thousands of iterations of "goofy and obviously erroneously evil AI", with enough time to take some action. And even then, current systems including this one are more often than not trained with predictive objective, which is very different compared to usually postulated reinforcement learning objective. Systems trained with prediction objective shouldn't be prone to becoming agents, much less dangerous ones.
If you read Scott's blog, you should remember the prior post where he himself pointed that out.
In my honest opinion, unaccountable AGI owners pose multiple OOM more risk than alignment failure of a hypothetical AI trying to predict next token.
We should think more about the Human alignment problem.
We're making it in our image. Literally.
Human social good isn't some inherent thing to the biology of the brain. There are aspects like mirror neurons and oxytocin that aid its development, but various "raised by wolves" case studies have shown how damaging not having exposure to socialization information during developmental periods of neuroplasticity is on humans and later integration into society.
We're building what's effectively pure neuroplasticity and feeding it almost all the data on humanity we can gather as quickly as we can.
What comes out of that is going to be much more human than a human child raised by dogs or put in an isolation box.
Don't get so caught up in the body as what makes us quintessentially human. It's really not.
For example, some total whack job of an authoritarian leader is in charge of a sufficient nuclear arsenal and decides to intimidate an adversary by destroying a couple minor cities, and the situation escalates badly. (stupidity)
Or we finally pollute our air and/or water with a persistent substance that either greatly reduces human life span or reproduction rate. (hubris)
I think either of the above is more likely to occur, and I am not commenting on current world events in any way. I think when something bad finally happens, it is going to come completely out of left field. Dr Strangelove style.
And the last of us will be saying "Hmmm, I didn't see that coming".
Our desire for purpose is a delusion.
Deep learning is very good at things we can do without thinking, and is in some cases superhuman in those tasks because it can train on so much more data. If you look at the list of tasks in System 1 vs System 2, SOTA Deep learning can do almost everything in System 1 at human or superhuman levels, but not as many in System 2 (although some tasks in System 2 are somewhat ill-defined), System 2 builds on system 1. Sometimes superhuman abilities in System 1 will seem like System 2. (A chess master can beat a noob without thinking while the noob might be thinking really hard. Also GPT-3 probably knows 2+2=4 from training data but not 17 * 24, although maybe with more training data it would be able to do math with more digits 'without thinking' ).
System 1 is basically solved, but System 2 is not. System 2 could be close behind System 2 by building on System 1 but it isn't clear how long that will take.
[1]. https://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow#Summar...
This is happening since a few months ago:
Wei et al (2022). Chain of Thought Prompting Elicits Reasoning in Large Language Models. https://arxiv.org/abs/2201.11903
As far as we see, the https://www.gwern.net/Scaling-hypothesis continues to hold, and critics have to move their goalposts every year or two.
yes, and there is an insight here that I think is often missed in the popular framing of AI x-risk: the autonomous systems we have today (which, defined broadly, need not be entirely or even mostly digital) are just as vulnerable to this
the AGI likely to pose extinction risk in the near term has humans in the loop
less likely to look like Clippy, more likely to look like a catastrophic absence of alignment between loci of agency (social, legal, technical, corporate, political, etc)
And for this to develop in machines, machines would have to be subject to many mistakes along the way leading to all kinds of outcomes that we hold humans accountable for by fining them, sending them to jail, some of them dying etc. I think that would be so wholly unpalatable to man kind they'd cut that experiment short before it ever reached any sort of scale.
I agree with your conclusion that enough of the rules can't be encoded by us as we don't even know them and for machines to acquire them the traditional way is, I believe, fundamentally disagreeable to humans.
I've been pondering about this problem for a while now[1], Could we build a collective intelligence through community submitted recipes for second-order decisions for various common activities via generalized schema?
I didn't think about addressing this for AI, But as an aid for us in multi-order thinking. But now that you mention it to be a barrier for AGI, It does make sense.
[1] 'Plan Second-Order, Third-Order consequences': https://needgap.com/problems/263
Here's a real generalist question? What's the point of conversation? In the sense that its a "social game", what are the strategy sets on a given turn, and what does it even mean to "win"? Forget about Artificial Bullshiters vs Artificial Idea-what-to-say. How can we even speculate if ABS or "real AI" solves the problem better when we don't really specify the problem space or how to recognize a solution?
In terms of calls and appropriate responses and responses to responses and terminal states, what even is conversation at the protocol level?
For example, I wonder what expectations we will set for ourselves when we pick up a pencil once it becomes common knowledge that production-ready digital art can be created by an AI within minutes. What will people be saying about "old art" in those days? What will we think of people that deliberately choose to disavow AI art?
It is already happening, but it is not a new phenomenon - just the prosecution of the effects of widespread inadequate education.
The undesirable effect namely is, an increase of overvaluing cheap, thin products - a decrease in recognizing actual genius, "substance". For example, some seem to be increasingly contented with "frequentism" as if convinced that language is based on it - while it is of course the opposite - one is supposed to state what is seen, and to state "plausible associations" would have been regarded as a clear sign of malfunction. There seems to be an encouragement to take for "normal" what is in fact deficient.
Some people are brought to believe that some shallow instinct is already the goal and do not know "deep instinct", trained on judgement (active digestion, not passive intake).
The example provided fits:
> production-ready digital art
For something to be artistic it has to have foundational justifications - a mockery of art is not art. The Artist chose to draw that line under a number of evaluations that made it a good choice, but an imitator, even in a copy, used the ("frequentist", by the way note) evaluation that the other is an Artist - and there is a world of depth difference between the two.
The difference is trivial, and yet, many are already brought to confuse the shallow mockery and the deep creation.
For a).2). you argued in favor of symbolic reasoning. My personal interpretation of that line of thinking is: it helps for us to understand complex thinking machines, but it is not necessarily the building block for the thinking machine.
In the area of AI, there are a lot of opinions, but at the end of the day, builders win the argument. So far, what builders have shown gives me optimistic hope.
I'd like to offer some perspective as a layman who jumps on "imminent AGI": that's what these AI folks are trying hard to make me think.
It's like the research papers that say there's an "imminent cure to cancer"
My own effort, which focusing on natural language understanding: https://lxagi.com
Deleted Comment
I increasingly wonder at what point we'll realize that humans aren't actually particularly good at what we've specialized into (there's just not much competition), and our failure to picture what 'better' looks like may be less about the impossibility of better to exist than it is the impossibility of it for humans to picture it.
I keep seeing predictions of what will be impossible for AI because it treads on our perceived human uniqueness (i.e. sure AI beat a human at chess but it'll be 25+ years before it will beat us at Go) needing to get walked back, and yet we continue to put forward a new iteration of that argument at every turn.
Maybe AI will turn out to be better at identifying what's good for humanity than humanity is. Because frankly, humanity is downright awful at that skill and has been for pretty much its entire existence.
(a) I've noticed a common trend of AI researchers looking at the tree in front of them and saying "well, this tree is not also a forest and won't be any time soon."
But there's not always awareness of what's going on in other specialized domains, so an AI vision researcher might not be intimately aware of what's currently being done in text or in "machine scientists" in biology for example.
As well, it overlooks the development of specialization of the human brain. We have some specialized structures that figured their niche out back with lizards, and others that developed much later on. And each of those specialized functions work together to give rise to 'human' intelligence.
So GPT-3 might be the equivalent of something like the Wernicke's area, and yes - on its own it's just a specialized tool. But what happens as these specialized tools start interconnecting?
Throw GPT-3 together with Dall-E 2 and the set of use cases is greater than just the sum of the parts.
This is going to continue to occur as long as specialized systems continue to improve and emerge.
And quickly we'll be moving into territory where orchestration of those connections is a niche that we'll both have data on (from human usage/selection of the specialist parts) and will in turn build meta-models to automate sub-specialized models from that data.
Deterministic reasoning seems like a niche where a GAN approach will still find a place. As long as we have a way for one specialized model to identify "are these steps leading to X" we can have other models only concerned with "generate steps predicted to lead to X."
I don't think we'll see a single model that does it all, because there's absolutely no generalized intelligence in nature that isn't built upon specialized parts anyways, and I'd be surprised if nature optimized excessively inefficiently in that progress.
Will this truly be AGI in a self-determining way? Well, it will at least get closer and closer to it with each iteration, and because of the nature of interconnected solutions, will probably have a compounding rate of growth.
In a theoretical "consciousness" sense of AGI, I think the integrated information theory is interesting, and there was a paper a few years ago about how there's not enough self-interaction of information possible in classical computing to give rise to consciousness, but we'll probably have photonics in commercial grade AI setups within two years, so as hand-wavy as the IIT theory is, the medium will be shifting towards one compatible with their view of consciousness-capable infrastructure much sooner for AI than quantum competing in general.
So I'd guess we may see AI that we're effectively unable to determine if it is "generally intelligent" or 'alive' within 10-25 years, though I will acknowledge that AI is the rare emerging tech that I've been consistently wrong about the timing on in a conservative direction (it keeps hitting benchmark improvements faster than I think it will).
(b) The notion AGI will have it out for us is one of the dumbest stances and my personal pet peeves out there, arguably ranked along with the hubris of "a computer will never be able to capture the je ne sais quoi of humanity."
The hands down largest market segment for AI is going to be personalization, from outsourcing our work to a digital twin of ourselves to content curation specific to our own interests and past interactions.
Within a decade, no one is going to give the slightest bit of a crap about interactions with other humans in a Metaverse over interacting with AIs convincingly human enough but with the key difference of actually listening to our BS rather than just waiting for their turn to talk.
There's a decent chance we're even going to see a sizable market for feeding social media data of deceased loved ones and pets into AI to make twins available in such settings (and Microsoft already holds a patent on that).
So do we really think humans are so repugnant that the AI which will eventually reach general intelligence within the context of replicating itself as ourselves, as our closet friends and confidants, as our deceased loved ones - will suddenly decide to wipe us out? And for what gains? What is AI going to selfishly care about land ownership and utilization for?
No. Even if some evolved AGI somehow has access to DARPA killer drones and Musk's Terminator robots and Boston Dynamics' creepy dogs, I would suspect a much likelier target would be specific individuals responsible for mass human suffering the AIs will be exposed to (pedophiles, drug kingpins, tyrants) than it is grandma and little Timmy.
We're designing AI to mirror us. The same way some of the current thinking of how empathy arises in humans is from our mirror neurons and the ability to put ourselves in the shoes of another, I'm deeply skeptical of the notion that AI which we are going to be intimately having step into human shoes will become some alien psychopath.
It’s a very entertaining read from a couple of years ago (I think I’ve read it in 2017), and man, have things happened in the field since then. If feels like things truly start coming together. Transformers and then some incremental progress look like a very, very promising avenue. I deeply wonder in which areas this will shape the future more than we are able to anticipate beforehand.
Even the best-case scenario that some are describing, of uploading ourselves into some kind of post-singularity supercomputer in the hopes of being conscious there, doesn't seem very far from plain extinction.
And that's OK. We are one step toward the universe understanding itself, but we certainly aren't the final step.
So far, we have task-driven AI/ML. It solves a problem you tell it to solve. Then you, as the engineer, need to make sure it solves the problem correctly enough for you. So it really still seems like it would be a human failing if something went wrong.
So I'm wondering why there is so much concern that AI is going to destroy humanity. Is the theoretical AI that's going to do this even going to have the actuators to do so?
Philosophically, I don't have an issue with the debate, but the "AI will destroy the world" side doesn't seem to have any tangible evidence. It seems to me that people seem to take it as a given that it's possible AI could eliminate all of humanity and they do not support that argument in the least. From my perspective, it appears to be fearmongering because people watched and believed Terminator. It appears uniquely out-of-touch.
If we manage to make a 'better' replacement for ourselves, is it actually a bad thing? Our cousin's on the hominoid family tree are all extinct, yet we don't consider that a mistake. AI made by us could well make us extinct. Is that a bad thing?
For me immortality a bigger thing than the teraflops. Also I don't think regular humanity would be got rid of but continue in parallel.
We should ask our compute overlords to perform their experiments in as open environment as possible, just because we, the public, should have the power to oversee the exact direction this AI revolution is taking us.
If you think about it, AI safetyism is a red herring compared to a very real scenario of powerful AGIs working safely as intended, just not in our common interest.
The safety of AGI owners' mindset seems like a more pressing concern compared to a hypothetical unsafety of a pile of tensors knit together via gradient descent over internet pictures.
> Build a computer that can multiply two ten-digit numbers in a split second—incredibly easy. Build one that can look at a dog and answer whether it’s a dog or a cat—spectacularly difficult. Make AI that can beat any human in chess? Done. Make one that can read a paragraph from a six-year-old’s picture book and not just recognize the words but understand the meaning of them? Google is currently spending billions of dollars trying to do it. Hard things—like calculus, financial market strategy, and language translation—are mind-numbingly easy for a computer, while easy things—like vision, motion, movement, and perception—are insanely hard for it.
The children's picture book problem is solved; those billions of dollars were well-spent after all. (See, e.g., DeepMind's recent Flamingo model [1].) We can do whatever we want in vision, more or less [2]. Motion and movement might be the least developed area, but it's still made major progress; we have robotic parkour [3] and physical Rubik's cube solvers [4], and we can tell a robot to follow simple domestic instructions [5]. And Perceiver (again from DeepMind [6]) took a big chunk out of the perception problem.
Getting a computer to carry on a conversation [7], let alone draw art on par with human professionals [8], weren't even mentioned as examples, so laughably out of reach they seemed in the heathen dark ages of... 2015.
And as for recognizing a cat or a dog — that's a problem so trivial today that it isn't even worth using as the very first example in an introductory AI course. [9]
If someone re-wrote this post today, I wonder what sorts of things would go into the "hard for a computer" bucket? And how many of those would be left standing in 2029?
[1] https://arxiv.org/abs/2204.14198
[2] https://arxiv.org/abs/2004.10934
[3] https://www.youtube.com/watch?v=tF4DML7FIWk
[4] https://openai.com/blog/solving-rubiks-cube/
[5] https://say-can.github.io/
[6] https://www.deepmind.com/open-source/perceiver-io
[7] https://arxiv.org/abs/2201.08239v2
[8] https://openai.com/dall-e-2/
[9] https://www.fast.ai/
Last time I checked - though it's been a long while I could not check thoroughly owing to other commitments - "«recognizing»" there was "consistently successfully guessing", not "critically defining". It may be that the problem was solved in the latest years, I cannot exclude it - but I have not seen around in the brief "news checking" exercise the signals required for the solution.
The real deal is far from trivial.
A clock can tell the time but does not know it.
For example: "Life is just chemistry", "Earth is just a pile of atoms", "Behaviours are just Turing Machines", etc.
I mean have you heard the word salad that comes out of so many people's mouths? (Including myself, admittedly)
Now consciousness on the other hand is unfathomable and (in its finitude) extremely depressing for me.
Date Weakly General AI is Publicly Known: https://www.metaculus.com/questions/3479/date-weakly-general...
(I really like the framing of "weakly general AI" since it puts the emphasis on the generality and not whether it's a superintelligence)
Edit: Probably not today, but mostly because 1.2B parameters isn't enough to get it the high winograd scores that PaLM etc have. But it seems pretty clear you could scale this architecture up and it will likely pass. The question is when someone will actually train a model that can do it
That's not to say it's not an important step. Showing that you can train one model on all of these disparate tasks at once and not have the system completely collapse is a big deal. And it lays the foundation for future efforts to raise the performance from "not totally embarrassing" to "human level". But there's still a ways to go on that front.
This system lacks that feature.
It's a curious concept, well illustrated in the novel Blindsight by Peter Watts. I won't spoil anything here but I'll highly recommend the book.
The only thing that comes to mind is how many different things come to mind to people when the term "superintelligence" is used.
The thing about this imagination process, however, is that what people produce is a "bag of capacities" without a clear means to implement those capacities. Those capacities would be "beyond human" but in what direction probably depends on the last movie someone watched or something similarly arbitrary 'cause it certainly doesn't depend on their knowledge of a machine that could be "superintelligent", 'cause none of us have such knowledge (even if this machine could go to "superintelligence", even our deepmind researchers don't know the path now 'cause these are being constructed as a huge collection of heuristics and what happens "under the hood" is mysterious to even the drivers here).
Notably, a lot of imagined "superintelligences" can supposedly predict or control X, Y or Z thing in reality. The problem with such hypotheticals is that various things may not be much more easily predictable by an "intelligence" than by us simply because such prediction involves imperfect information.
And that's not even touch how many things go by the name "consciousness".
Consciousness is defined as an entity that has an ethical framework that is subordinated to it's own physical existence, maintaining that existence, and interfacing with other conscious entities as if they also have an ethical framework with similar parameters who are fundamentally no more or less important/capable than itself.
Contrast with non-conscious super-intelligence that lacks physical body (likely distributed). Without a physical/atomic body and sense data it lacks the capacity to empathize/sympathize as conscious entities (that exist within an ethical framework that is subordinated to those limitations/senses) must. It lacks the perspective of a singular, subjective being and must extrapolate our moral/ethical considerations, rather than have them ingrained as key to it's own survival.
Now that I think about it, it's probably not much different than the relationship between a human and God, except that in this case it's: a machine consciousness and a machine god.
To me, the main problem is that humans (at large) have yet to establish/apply a consistent philosophy with which to understand our own moral, ethical, and physical limitations. For the lack of that, I question whether we're capable of programming a machine consciousness (much less a machine god) with a sufficient amount of ethical/moral understanding - since we lack it ourselves (in the aggregate). We can hardly agree on basic premises, or whether humanity itself is even worth having. How can we expect a machine that we make to do what we can't do ourselves? You might say "that's the whole point of making the machine, to do something we can't" but I would argue we have to understand the problem domain first (given we are to program the machine) before we can expect our creations to apply it properly or expand on it in any meaningful way.
Life has no purpose so clearly there is no rational reason to continue living/existing. A super-rational agent must know this.
I think that intelligence and emotions, in particular fear of death or desire to continue living, must evolve in parallel.
That sentence should have been "try to imagine consciousness".
Intelligence is the system allowing the redefinition of ideas over the entities constituting an inner representation of a world: how a non-trivial system of "consciousness" (otherwise the use of the term would be a waste) could have to be employed for that?
1. tell me about a cat (given a prompt such as "describe a cat to me")
2. recognize a cat in a photo, and describe the cat in the photo
does the model understand that a cat that it sees in an image is related to a cat that it can describe in natural language?
As in, are these two tasks (captioning an image and replying to a natural language prompt) so distinct that a "cat" in an image excites different neurons than a "cat" that I ask it about? Or is there overlap? Or we don't know :)
I wonder if you could mix the type of request. Like, provide a prompt that is both text and image. Such as "Here is a picture of a cat. Explain what breed of cat it is and why you think so." Possibly this is too advanced for the model but the idea makes me excited.
https://twitter.com/serkancabi/status/1519697912879538177/ph...
I don't know exactly how this particular model works but it is creating cross modal relationships otherwise it would not have the capacity to be good at so many tasks.
Not really. Embeddings from images occupy a different region of the space than embeddings from text. A picture of cat and the text "cat" do not resolve to the same embedding.
This is why DALL-E has a model learning to translate CLIP text embeddings into CLIP image embeddings before decoding the embedding.
Gato apparently just uses a single model.
Similar to the so-called "Jennifer Aniston neurons" in humans that activate whenever we see, hear, or read a particular concept: https://en.wikipedia.org/wiki/Grandmother_cell
Moreover, can it relate non-cat but cat-like objects to it's concept of cattyness? As in, this is like a cat because it has whiskers and pointy ears, but is not like a cat because all cats I know about are bigger than 10cm long. It also doesn't have much in the way of mouseyness: it's aspect ratio seems wrong.
Example: https://old.reddit.com/r/dalle2/comments/u9awwt/pencil_sharp....
We only have very limited (but suggestive) evidence that the human brain has abstract "cat" neurons involved in various sensory and cognitive modes. Last time I paid attention, there was reasonably strong evidence that an image of a cat and reading the word cat used some of the same neurons. Beyond that it was pretty vague, there seemed to be evidence of a network that only weakly associates concepts from different modes, which isn't consistent with most people's subjective experience.
But then we have other evidence that what we think of as our subjective cognitive experience is at least partly a post-hoc illusion imposed for no apparent reason except that it creates an internal narrative consistency (which presumably has some utility, possibly in terms of having a mind others can form theories about more readily).
This is rather mind blowing. Does it also mean that the generalist network is smaller than the sum of all specialist networks that are equivalent? Even if not, I find the idea that a single network can be used for such diverse tasks at all highly fascinating.
The interesting questions imo, which they studied, is what kind of added generalization takes place by learning across the different tasks. For example, does learning multiple tasks make it better at a given task than a model that is just trained for one task, and can it generalize to new tasks (out of distribution).
They looked at how it performed on held out tasks (see fig 9 in the paper). I'm still getting my head around the result though so couldn't summarize their finding yet.
Edit: the paper is here https://storage.googleapis.com/deepmind-media/A%20Generalist...
There is currently another submission on the front page that links to it directly.
This seems myopic.
Our own brains have specialized areas. Someone who has a stroke impacting Broca's region isn't going to have a fun time trying to do what that region was specialized for.
And as a society, I don't think we'd have gotten very far if every individual was expected to perform every task.
Where generalization in AI seems like the holy grail is not in replacing more specialized models, but in orchestrating them.
As impressive as this effort at generalization is, unless the work here is going to translate into orchestration and management of other specialized models, I don't think this is going to have much market relevance unless the generalized approach can somehow (in that generalization) gain an advantage in outperforming specialized approaches.
https://pubmed.ncbi.nlm.nih.gov/16437554/
There are a number of structures related to specific functionality - facial recognition is similarly hardwired.
Problems with the fusiform face area lead to prosopagnosia, it'll also be the region of the brain responsible for the hollow face illusion.
But Intelligence is one of those, and when people start calling naïvely calling "intelligence" its opposite "imprint", there is a problem. There are even the seeds of a social problem.
> orchestrating ... specialized models
Intelligence (ontology development) is one such specialized model which I have seen more faithfully and consciously attempted in classic AI as opposed to contemporary. Orchestration is crucial for the peripherals that allow the input/output. The study of "generalization" is none the less crucial, because we are to investigate the development of a system that can intrinsically, by its own nature, be exported to the largest number of relatable skills. Even with the highest specialization, up to physical, there is a matter of organicity: a hand is "specialized", yet a hook is not a hand.
At the moment, it seems like the model needs to be trained with each modality of data in mind at the start, but a generalised "supermodality" that can deliver all the others would allow truly generalised learning if the model were still capable of making sense of the input.
You'd obviously still need to finetune on any new modalities, but you wouldn't need to start from scratch.