I really like that people asking fundamental questions about the way AI can be pushed forward beyond just ML seems to gain a little bit of traction again, at least in the public eye. Even Kissinger opined on the topic just a few days ago
I agree with Pearl that there's something deeply misguided about thinking that intelligence truly can be solved as a data optimization problem. I'd be happy if we'd see more research about AI at a symbolic level again.
The biggest hindrance seems to be funding. ML is really successful in a lot of commercial domains, general AI is a big moonshot. With most researchers moving from long term university positions to the business sector I'm concerned about this sort of research, not just in computer science.
> I agree with Pearl that there's something deeply misguided about thinking that intelligence truly can be solved as a data optimization problem.
When all you have is a hammer... I think we’re really thinking about the whole thing backwards, as if intelligence only comes from the cortex of the brain. I think human level intelligence is fundamentally an emotional state of being - our core lizard brain values aren’t something to be swept aside - they’re a fundamental part of real intelligence. It’s like trying to build a car starting from the body and electronics rather than the motor.
Well, if a computer scientist only has a bunch of data, is there any conceivable way to reason about that data that doesn't ultimately boil down to a data optimization problem?
I hear that correlation does not imply causation but also that you can't distinguish correlation from causation merely with a stream of data. Is there any way out of this situation with just data?
One thing that I never see brought into the equation but I believe is fundamental to human "general" intelligence is emotional response. It's hugely instrumental in terms of us deciding what's important to investigate, which input to analyze and how to mold our output. The importance of emotion in guiding and driving our learning is most evident in children, during the first four or five years of life of development.
I think a huge leap forward in the field could be made by modeling an emotional system similar to ours. It also seems like it would be an incredibly difficult and fuzzy problem to solve.
Addendum: After thinking about it for a few minutes more, I bet an implicit understanding of cause and effect could be derived by a system designed to try to optimize it's emotional state
Not just emotion, but also aversion to certain physical states - such as hunger or pain.
Because of that, I think a "true AI" - at least one without a simulated humanlike body - would have rather different desires. Optimization towards those desires rather than humanlike desires would likely result in something that distinctly does not act human.
I can't help but wonder if we've ignored or disregarded any first steps to a general AI because the resulting building blocks didn't match any of our expectations from real-world models of instinct/intelligence.
I do think the impact of emotions on our thinking is understated, but I also think that focusing on this aspect is going at it backwards. A value system is necessary for emotions, otherwise happy and sad mean nothing. I think this is what you mean when you talk about "modeling emotions". At the same time, some intelligence seems like a prerequisite as blindly following a value system will lead to massive hedonistic AI orgies. Er, I mean, an override mechanism is required, and intelligence is the best possible solution, so we're back at the fundamental problem. I think there is a lot more intelligence behind even our most primitive urges than we realize. Hope that made sense, typing on mobile.
Isn't this the theory of co sciousness presented in After On? I am curious what people are formally studying emotional theories of co sciousness. I'd love to read more.
It could start with a value system rather than raw emotions - at some point it might decide that it doesn’t really want to do some work. The idea of controllability in all senses I think complicates the issue, but it’s definitely a necessary safety net.
More than funding, then issue is data. Untangling correlation from causation usually requires running additional experiments.
It's important for people to understand these limits of current ML, but they are happening for a good reason. You can apply associative reasoning in many environments that don't support causal reasoning, and we are still scratching the surface of the value discovering associations can create.
I don't disagree. However, the next time you find yourself pondering the question "Why did Bob do X? That's so bizarre, and it makes no sense", you should consider what the repercussions may be if Bob were not just your awkward acquaintance but rather an autonomous computer system distributed across the globe that's assigning credit scores to people. Is there really room for 'Well it just felt right at the time' in those situations?
Humans are often quite bright, but we're also known to do the wrong thing for no discernible reason. This is to be expected when there's no fundamental formal system behind behavior, and behavior is instead driven by a black-box neural network.
There's problems with that attitude. First, it offers no scientific insight. We don't learn anything about the fundamental nature and skeleton of minds by simply replicating evolutionary processes. It does not grant us knowledge.
The equivalent in engineering would be to throw over trees and rocks in the hope of building a bridge. Clearly, that is unsatisfactory, we strive to understand the meaning of systems so that we can reason about them and alter them in predictable and fundamental ways.
Secondly, we don't know how likely it is that evolution produces intelligence. Maybe we're the only intelligent spot in the universe and it's an aberration. It took 4 billion years as well.
That seems to be a fundamentally impoverished way to go about things. We shouldn't forego the ability to understand minds at a deep level just because we have made practical strides in closed domains. That would be to mistake a trojan horse for an actual horse.
What was the data optimization problem? To pass our genes on? Because lots of critters have been doing it far longer than the big brained ones. Bacteria and viruses are more successful than any life forms, and they're not running any neural equivalent of data optimization.
Nature has a huge environment to run its agents in. AI can only access games. Robots IRL are too expensive for optimisation. Solution: better simulators.
Worth noting that there has been a fair bit of good research in causal machine learning in the last year or so, for example "Implicit Causal Models for Genome-wide Association Studies" (https://arxiv.org/pdf/1710.10742.pdf).
The key point of this paper is that neural networks really are very good at "curve fitting" and that this curve fitting in the context of variational inference has advantages for causal reasoning, too.
Neural networks can be used in a variety of structures, and these structures tend to benefit from the inclusion of powerful trainable non-linear function approximators. In this sense, deep learning will continue to be a powerful tool despite some limitations in its current use.
I think Pearl, who's obviously remained very influential for many practitioners of machine learning, knows the value of "curve fitting". However I think it's a bit hard for a brief interview to sit down and have a real conversation about the state of the art of an academic field and the "Deep Learning is Broken" angle is a bit more attractive.
It's worth considering that anywhere in graphical models where coefficients of any sort learned can be augmented by neural networks (such as in the last decade of natural language processing, where the SOTA of almost all problems has been successfully neuralized).
I wonder if Deep Belief Machines and their flavor of generative models, which seem closer in nature to Pearl's PGMs, have a chance to bridge the gap involved.
Edit, as an aside: Given the enormously high dimensionality of personal genomes and the incredibly small sample size, for over a decade I've failed to put any trust in GWAS studies and found my suspicion supported on a number of occasions, considering difficulty in reproducibility likely brought about by the above problem. Is there any reason to think that improved statistical methods can possibly surmount the fundamental problem of limited sample size and high dimensionality?
Numerous important biomedical findings have resulted from GWAS. Most GWAS today are inherently reproducible since their hits usually come from multi-stage designs with independent samples. Sample sizes are no longer "incredibly small" either; large GWAS often have in the order of 100s of 1000s of patients. Some have over a million.
I suppose the most important idea is that GWAS aren't really supposed to show causality. "Association" is in the name. GWAS are usually hypothesis generating (e.g., identification of associated variants) and then identified variants can be probed experimentally with all of the tools of molecular biology.
In summary, GWAS have their problems, but I think your statement is a bit too strong.
There are a lot of efforts in developing models that understand causal relationships within mainstream machine learning community. Mostly to train models that don't require a lot of training examples. Deep learning usually requires a lot of data and trained models are not easily transferable to other tasks. Yet humans tend to transfer their knowledge from other tasks pretty easily to seemingly unrelated tasks. This seems to be due to our mental models surrounding causal relationships. One example of such efforts is schema networks. It is a model-based approach to RL that exhibits some of the strong generalization abilities that can be key to human-like general intelligence. https://www.vicarious.com/2017/08/07/general-game-playing-wi...
I can't help but see this as another example of the pattern in which a big name in a field gets up and says that the current direction of their field (deep learning) is great and all but not really making progress on the big question (intelligence), and that to solve that question we need to solve another big question (what is causality) before we can make true progress. Other examples to me are Chomsky on consciousness and its implications for language, Einstein on causality w.r.t. quantum theory. This isnt to say the big name is wrong, just to point out a potential pattern.
In history we call this determinism. The more you know about a historic choice and the complex mechanisms around it, the more it makes perfect sense while at the same time leaving you absolutely clueless about the why.
Christianity being chosen by the Roman Empire is the typical example. To most people the choice makes perfect sense, because we look back at what it brought with it. But when you put yourself in the heads of the decision makers and look at all the options they had, well, it makes no sense at all.
A lot of machine learning tells us trends, but it tells us nothing about the why, and I completely agree with the article about how useless that data is. I mean, it’s probably great at harmless things, but when my elaborate online profile still can’t figure out why I happen to read a cultural, artsy but somewhat conservative news paper, despite the fact that my data shows the algorithm that I really really shouldn’t be doing that, well, then we simply can’t use ML for any form of decision making or even as an advisor. At least not in the public sector.
Yeah, I think it's worth also asking whether humans /actually/ are any good at answering the 'why' with anything but bullshit. I would argue that we're pretty good at understanding causality in very limited circumstances (the window broke because that kid threw the ball), and extremely overconfident in our ability to understand causation in a much broader range of circumstances (the stock price went up because...). This overconfidence drives a lot of the decisions we make, for better or for worse.
It's an area where if we push hard on AI, we'll likely have to come to terms with how bad we are in this area, and ask ourselves whether we feel comfortable deploying 'thinking machines' with similar levels of incompetence and/or arrogance.
Pearl's words from the Introduction of "BAYESIANISM AND CAUSALITY, OR, WHY I AM ONLY A HALF-BAYESIAN":
"I turned Bayesian in 1971, as soon as I began reading Savage’s monograph The Foundations of Statistical Inference [Savage, 1962]. The arguments were unassailable: (i) It is plain silly to ignore what we know, (ii) It is natural and useful to cast what we know in the language of probabilities, and (iii) If our subjective probabilities are erroneous, their impact will get washed out in due time, as the number of observations increases.
Thirty years later, I am still a devout Bayesian in the sense of (i), but I now doubt the wisdom of (ii) and I know that, in general, (iii) is false."
Subjective probabilities are based on the model. Increasing observations won't help if you have the wrong model to begin with. So we need causal methods to ask if the model is correct. We also need methods to propose new models or rebuild if it is wrong as well.
I bought Judea Pearl’s new book The Book of Why last night after reading this article. So far I love the book. I manage a machine learning team at work and I appreciate Pearl’s discussion of how deep learning and statistics won’t lead to strong AI.
When I saw another one of these publicity articles, I basically ran to buy the book. It's really nice to have a book that will help me get the intuition and history of causal modeling rather than just giving theorems about graph structure under intervention.
I agree, it is nice to have a very clear high level approach to causal reasoning. I find his other books to be ‘slow going’ so I hope that after reading the Why book, I will have an easier time absorbing his earlier work.
The core issue is trust, explanation is one part of trust, but there are deeper issues. After all, if I explain that I have made this clinical decision because it results in lower mortality and you point out that the mortality statistic is to shit, and then I point out that we can't do the experiment required to work out the mortality statistic properly because that would mean potentially killing children... we have an issue.
We trust doctors and pilots, they offer partial explanations that we can somewhat understand, but they are backed by experience and qualification. Their perspective is informed by science - some good, some bad. Most of us don't think about that.
We have a perspective based on our cultural and social background, the machine must understand this and provide alternative explanations to suit us.
I have written a long article on this all, but I can't finish the game theory off!
Readit News
https://www.theatlantic.com/magazine/archive/2018/06/henry-k...
I agree with Pearl that there's something deeply misguided about thinking that intelligence truly can be solved as a data optimization problem. I'd be happy if we'd see more research about AI at a symbolic level again.
The biggest hindrance seems to be funding. ML is really successful in a lot of commercial domains, general AI is a big moonshot. With most researchers moving from long term university positions to the business sector I'm concerned about this sort of research, not just in computer science.
When all you have is a hammer... I think we’re really thinking about the whole thing backwards, as if intelligence only comes from the cortex of the brain. I think human level intelligence is fundamentally an emotional state of being - our core lizard brain values aren’t something to be swept aside - they’re a fundamental part of real intelligence. It’s like trying to build a car starting from the body and electronics rather than the motor.
I hear that correlation does not imply causation but also that you can't distinguish correlation from causation merely with a stream of data. Is there any way out of this situation with just data?
I think a huge leap forward in the field could be made by modeling an emotional system similar to ours. It also seems like it would be an incredibly difficult and fuzzy problem to solve.
Addendum: After thinking about it for a few minutes more, I bet an implicit understanding of cause and effect could be derived by a system designed to try to optimize it's emotional state
Because of that, I think a "true AI" - at least one without a simulated humanlike body - would have rather different desires. Optimization towards those desires rather than humanlike desires would likely result in something that distinctly does not act human.
I can't help but wonder if we've ignored or disregarded any first steps to a general AI because the resulting building blocks didn't match any of our expectations from real-world models of instinct/intelligence.
It's important for people to understand these limits of current ML, but they are happening for a good reason. You can apply associative reasoning in many environments that don't support causal reasoning, and we are still scratching the surface of the value discovering associations can create.
Humans are often quite bright, but we're also known to do the wrong thing for no discernible reason. This is to be expected when there's no fundamental formal system behind behavior, and behavior is instead driven by a black-box neural network.
The equivalent in engineering would be to throw over trees and rocks in the hope of building a bridge. Clearly, that is unsatisfactory, we strive to understand the meaning of systems so that we can reason about them and alter them in predictable and fundamental ways.
Secondly, we don't know how likely it is that evolution produces intelligence. Maybe we're the only intelligent spot in the universe and it's an aberration. It took 4 billion years as well.
That seems to be a fundamentally impoverished way to go about things. We shouldn't forego the ability to understand minds at a deep level just because we have made practical strides in closed domains. That would be to mistake a trojan horse for an actual horse.
But even if it were true, do you want to spend billions of years solving it that way?
Dead Comment
The key point of this paper is that neural networks really are very good at "curve fitting" and that this curve fitting in the context of variational inference has advantages for causal reasoning, too.
Neural networks can be used in a variety of structures, and these structures tend to benefit from the inclusion of powerful trainable non-linear function approximators. In this sense, deep learning will continue to be a powerful tool despite some limitations in its current use.
I think Pearl, who's obviously remained very influential for many practitioners of machine learning, knows the value of "curve fitting". However I think it's a bit hard for a brief interview to sit down and have a real conversation about the state of the art of an academic field and the "Deep Learning is Broken" angle is a bit more attractive.
I wonder if Deep Belief Machines and their flavor of generative models, which seem closer in nature to Pearl's PGMs, have a chance to bridge the gap involved.
Edit, as an aside: Given the enormously high dimensionality of personal genomes and the incredibly small sample size, for over a decade I've failed to put any trust in GWAS studies and found my suspicion supported on a number of occasions, considering difficulty in reproducibility likely brought about by the above problem. Is there any reason to think that improved statistical methods can possibly surmount the fundamental problem of limited sample size and high dimensionality?
I suppose the most important idea is that GWAS aren't really supposed to show causality. "Association" is in the name. GWAS are usually hypothesis generating (e.g., identification of associated variants) and then identified variants can be probed experimentally with all of the tools of molecular biology.
In summary, GWAS have their problems, but I think your statement is a bit too strong.
Deleted Comment
Dead Comment
Christianity being chosen by the Roman Empire is the typical example. To most people the choice makes perfect sense, because we look back at what it brought with it. But when you put yourself in the heads of the decision makers and look at all the options they had, well, it makes no sense at all.
A lot of machine learning tells us trends, but it tells us nothing about the why, and I completely agree with the article about how useless that data is. I mean, it’s probably great at harmless things, but when my elaborate online profile still can’t figure out why I happen to read a cultural, artsy but somewhat conservative news paper, despite the fact that my data shows the algorithm that I really really shouldn’t be doing that, well, then we simply can’t use ML for any form of decision making or even as an advisor. At least not in the public sector.
It's an area where if we push hard on AI, we'll likely have to come to terms with how bad we are in this area, and ask ourselves whether we feel comfortable deploying 'thinking machines' with similar levels of incompetence and/or arrogance.
"I turned Bayesian in 1971, as soon as I began reading Savage’s monograph The Foundations of Statistical Inference [Savage, 1962]. The arguments were unassailable: (i) It is plain silly to ignore what we know, (ii) It is natural and useful to cast what we know in the language of probabilities, and (iii) If our subjective probabilities are erroneous, their impact will get washed out in due time, as the number of observations increases.
Thirty years later, I am still a devout Bayesian in the sense of (i), but I now doubt the wisdom of (ii) and I know that, in general, (iii) is false."
We trust doctors and pilots, they offer partial explanations that we can somewhat understand, but they are backed by experience and qualification. Their perspective is informed by science - some good, some bad. Most of us don't think about that.
We have a perspective based on our cultural and social background, the machine must understand this and provide alternative explanations to suit us.
I have written a long article on this all, but I can't finish the game theory off!
But then.. I guess that's the point!