68 | Melanie Mitchell on Artificial Intelligence and the Challenge of Common Sense

0:00:00 Sean Carroll: Hello everyone and welcome to the Mindscape Podcast. I'm your host, Sean Carroll. We all know that artificial intelligence is an important thing. We've talked about it here on Mindscape. I'm sure we're gonna keep talking about it down the road. We also know that it faces some obstacles. I don't mean the kinds of ethical obstacles we've talked about before; what should your self-driving car do? I mean obstacles in creating what would truly be considered a human level artificial intelligence, there are certain things that computers, that AIs are really really good at, way better than any human being and there's certain other things, they still fall short at. And those things that artificial intelligence is not very good at can be lumped under the general heading of common sense. There are aspects of the world that seem completely obvious and intuitive to every human being from a very young age, and yet not only do computers not know them automatically, we even have trouble teaching them to computers.

0:00:57 SC: So today's guest is Melanie Mitchell a computer scientist and complexity theorist at Portland State University and the Santa Fe Institute. She's the author of several books that I really like, and that her new book is called Artificial Intelligence: A Guide for thinking humans. And it doesn't pretend to give the once and for all solution to this problem, but it lays out exactly this issue of how artificial intelligence works, and why common sense as important and as basic as it is, is one of the most difficult things to teach to a computer. We also talk about what the ramifications of this difficulty are for what AI is, where it's going, how it will be good and bad as we move into the future. So let's go...

[music]

0:01:48 SC: Melanie Mitchell, thanks for being on the Mindscape Podcast.

0:01:58 Melanie Mitchell: I'm glad to be here.

0:02:00 SC: So you have an advantage over other people who write popular books about artificial intelligence, I think which is that Douglas Hofstadter, was your PhD thesis advisor, right?

0:02:08 MM: That's right, that's right, yes.

0:02:10 SC: So that not only is awesome, because he's done a lot of influential things, most obviously author of Gödel, Escher, Bach, which probably like you got a lot of people interested in this subject matter, but he continues to be a thoughtful and eloquent person thinking about these issues. And so, you tell this wonderful story, of a visit to Google with Hofstadter early on in the book. Why don't we start off by setting the stage with that kind of anecdote.

0:02:34 MM: Right. So maybe five years ago, some of the AI people at Google invited Douglas Hofstadter who is a very well-known writer about AI, thinker, researcher, they invited him to come and meet with them, to talk about AI and thinking about how to push forward an AI. I was living on the west coast in Portland, and the meeting was in Mountain View, so he asked me if I wanted to come and I absolutely did. So, we all showed up there and got to the Google building and found our way to the conference room and Doug got up to speak and he basically started telling the Google engineers about how terrified he was about AI, how much he hated AI, how much he feared it, how much he loathed it. He was extremely passionate and they were just sitting there mouths agape.

0:03:32 SC: Not what they expected to hear.

0:03:33 MM: Thinking what the heck is going on? And they quizzed him. And he was... His fear is a little bit complicated. It's not just that AI is gonna take over.

0:03:50 SC: Right that's the traditional fear that we hear about.

0:03:51 MM: Yeah.

0:03:52 SC: So, he has a slightly subtle, more different fear.

0:03:54 MM: Yeah. His fear was more that, in some sense, all the things that he found the most profound and close to his heart about human intelligence that he's been thinking about it for decades and decades, extremely carefully and thoroughly, he is worried that these things will in some sense, be too easy to automate, that AI, with its what he thinks of as cheap tricks, will succeed in creating a person with a profundity of one of his heroes like Chopin, or Gödel or.

0:04:31 SC: Or Bach or whoever.

0:04:33 MM: Or Bach or whoever, right. And that that would just horrify him that the human soul, if you will, would be so easy to capture in a cheap machine.

0:04:43 SC: And is the horror in the cheapness? If it were hard to do, but we could still do it in a computer, would he be less horrified?

0:04:50 MM: I think he...

0:04:51 SC: Or is that we're just tricking ourselves into thinking that it's like that?

0:04:53 MM: Right. So, it's hard to speak for him, but my impression is that the harder it is, the better.

0:05:00 SC: Right.

0:05:01 MM: And...

0:05:02 SC: So we want to learn that human creativity is somehow extremely hard and profound.

0:05:06 MM: Right. So, he extensively talks about Chopin, that's one of his favorite composers, and feels one of the most profound sort of humans who ever lived, and he fears that something like Chopin's music could be created by an AI program that just used a lot of superficial heuristics and cheap tricks as he calls them.

0:05:35 SC: Some random numbers, match some patterns.

0:05:37 MM: Yeah.

0:05:37 SC: And then you have the genius of Chopin.

0:05:39 MM: Right. And that would just be the worst thing in the world to him.

0:05:42 SC: Well, and there was an example, right? There was a computer program that tried to mimic Chopin.

0:05:46 MM: Right, and did a too good a job. So there is a program written by a composer named David Cope who is also a programmer, who wrote this program called experiments in musical intelligence or EMI. EMI used a lot of AI techniques and sort of very relatively simple statistical pattern matching techniques. And it did a pretty good job of, with lots of data to go on, of composing in the style of various composers, including Chopin. And Doug even tried to play one of these pieces for a group at the Eastman School of Music. A bunch of music theorists and composers themselves, and they were fooled. They didn't know which piece was by the real Chopin and which piece was by EMI. So, that just terrified him. And then the Google engineers at this meeting were very certain that human level AI, of the kind that Doug was fearing was imminent in some way, was, at least, we were getting much closer to it with deep learning and all the data-driven approaches of today. And I was just very confused about this whole meeting that Doug Hofstadter was so terrified. The Google engineers were so glib about the idea that we were gonna have human level intelligence, which I find to be very unlikely.

0:07:23 MM: So I decided this would be, it kind of gave me the idea of trying to learn a lot more about AI. I work in my own narrow area, but to learn more about it as a whole, and to say like, "What is the state of AI?" We read stuff in the media that just seems to me to be outrageous hype. And some people think that we're very close to human level AI, some people think we're extremely far away. What is the real state of AI? So that's kind of the topic of my book. I was sparked by this meeting at Google and by Doug's terror, which surprised me.

0:08:00 SC: And you don't share the terror or the optimism really.

0:08:04 MM: That's right, I don't share either. And the book is kind of an exposition of what I found, where AI is today, sort of what... Then we have to ask: What do we mean by human-level intelligence?

0:08:17 SC: Absolutely, yeah.

0:08:17 MM: And that's ill-defined, we don't understand intelligence, we don't understand the brain, we don't understand the mind, if they are two separate things. And I think that AI is now at a state where people just don't agree. There's a huge amount of disagreement in the field.

0:08:34 SC: Well, I've noticed this, and there are people who are not professional AI researchers, but are famous public intellectuals, Stephen Hawking, Elon Musk, Bill Gates, people like that, who warn against AI coming in the future and being super intelligent and taking over. And they get a lot of pushback from people who spend their lives doing AI. But I don't find the pushback especially convincing, because you can be so immersed in the day-to-day difficulties that you miss the possibility of something big going on. So, that's what you're trying to sort of step back and ask about in the book.

0:09:06 MM: Exactly. Yeah. And what's interesting about AI, which might be different from your field, I don't know, but everybody has an opinion about it. Everybody has a "informed opinion" about it. Everybody thinks they know what intelligence is, and how hard it is for AI to get it. And we get a lot of people who are not in the field who have never worked in this area opining [chuckle] with great confidence about the future, which is very strange.

0:09:34 MM: And I think most, as you say, most people who are kind of on the ground working day-to-day don't agree with a lot of these more over-the-top predictions. But if you take someone like Ray Kurzweil, okay, who is the... Sort of promotes the idea of the singularity. Where he thinks that AI is going to become at the level of humans within the next 10 years. And then a billion times more intelligent than humans within 20 years or 30 years, or something. He would say, "Well, if you're sitting on an exponential curve," and that's his idea that progress in AI is on an exponential curve, "if you're sitting in the middle of an exponential curve, it doesn't look exponential to you. But it's about to get very, very steep." And...

0:10:28 SC: The nice thing about sitting on curves that don't look exponential is some of them might not be exponential.

[laughter]

0:10:32 MM: Exactly, but he has books full of experiential curves that try and make the argument that we are on an exponential curve. And to people out in the world, it kind of feels that way, 'cause there is so much progress that's been reported recently. It does feel like there's been exponential progress. And my book tries to look at that progress and ask sort of: How much progress has actually been made?

0:11:00 SC: Yeah. And I think that, well, just parenthetically, I think there are almost no exponential curves in nature, really exponential. The expansion of the universe is one of them, but most curves look exponential at the beginning and then flatten off, or decay away. So I don't think that the curve fitting tells us very much. Therefore, let's dig into some of the details. This is what your book does really, really well. Let's open up the hood a little bit and think about artificial intelligence. You mentioned all of the advances that people have seen recently. We can talk to Siri and so forth, it almost is like talking to a human. And this is based on, this is almost all based on deep learning and neural networks, is that a fair thing to say?

0:11:40 MM: Yes, absolutely. So deep learning and deep neural networks, which the definition of which is just neural networks with more than two layers.

[laughter]

0:11:50 SC: Oh, okay, I was wondering what the definition was, because I had seem them used interchangeably, but good.

0:11:54 MM: Yeah, so typically a neural network has, it has inputs and then it has what's called a hidden layer or an internal layer, and then it has an output layer. Those are sort of the traditional neural networks from say the 1990s that people use. So a deep network is a network with more than two hidden layers.

0:12:13 SC: That's very deep, okay.

0:12:14 MM: And the word deep should not be confused with the words it's deep sort of in like deep...

0:12:19 SC: Profound. [chuckle]

0:12:19 MM: Thinking, or profound, or you know. But it's a very good word, because it sounds...

0:12:28 SC: Deep.

0:12:28 MM: Very deep to people. But in any case, deep neural networks with a lot of data, big data to train them and very fast parallel computers to run them, have had a huge amount of success in some pretty narrow areas. Some of those areas are very useful. So it's like say, Siri, Speech Recognition, being able to transcribe our speech. That's been an amazingly successful sort of breakthrough due to deep learning. Object recognition to some extent has also become really successful and useful, or face recognition. Other kinds of computer vision have become very successful to deep learning. But, more and more people are beginning to see some of the limitations. So if you will... Yeah.

0:13:18 SC: Well, we'll get to the limitations but actually I wanna... But first, let's build our way up...

0:13:21 MM: Okay.

0:13:22 SC: What I should have said is, "Tell me about the perceptron."

0:13:23 MM: Oh, the perceptron.

0:13:24 SC: This is a wonderful little thing you talked about that led us eventually to neural networks.

0:13:28 MM: Right. So, the perceptron was a... Maybe the great-great-grandparent of today's deep neural networks, that was created in the 1950s by a psychologist named Frank Rosenblatt. He was looking at simple models of neurons. People have been making computer models of neurons ever since computers existed. That was one of the first things that people started to do. And, in that view, a neuron was a very simple device that gets input, if the input reaches some threshold, internal threshold, then the neuron fires.

0:14:06 SC: So we knew enough about neurons in the 1950s for him to be inspired by this to do a computer version?

0:14:10 MM: Exactly. Exactly. They were linear threshold devices, so to say. And, he found that if you create perceptrons, if you... He created them in hardware. So, you could see these in pictures of the perceptrons [laughter] that he created with these spaghetti masses of wires coming out, and, there's these giant main frame-looking computers.

0:14:30 SC: I forget, would transistors have been around by then, or was it vacuum tubes in his perceptron?

0:14:35 MM: Oh, that's a good question. I think...

0:14:37 SC: It was about that time I think. Yeah.

0:14:39 MM: Probably vacuum tubes. But they were large. [laughter] And they could do some things. And the thing about perceptrons that was so remarkable was that they could learn. They were one of the first systems based on the brain that could do some learning. And he developed an algorithm that allowed perceptrons to learn from examples.

0:15:03 SC: But it was really like one neuron was learning, it was not like... There were not huge networks.

0:15:06 MM: Yeah, essentially, or some network of simple... Some simple network of a group of neurons.

0:15:10 SC: Okay. So there were some simple networks, yeah.

0:15:11 SC: Yeah. There were some simple networks. And, they would learn. And, they could learn to do things like recognize handwritten characters; to some extent not very well, not as well as humans, but, Rosenblatt made a lot of promises, as people in AI tend to do. It helps to get funding. And, he made a lot of promises for what perceptrons would be able to do in the future. And, all of that was stomped upon by Marvin Minsky and Seymour Papert in around 1970 when they did a mathematical study of perceptrons, and showed that perceptrons were actually very limited in principle, in what they could learn. Even if you have as much data as you want to train them, they still cannot learn a lot of things that we want them to do.

0:16:09 SC: And the idea, just to make this absolutely clear to the audience, the idea was that just like a real neuron, there are inputs from the equivalent of dendrites. Right?

0:16:18 MM: Right.

0:16:18 SC: So, there's some wires going into the perceptron that say, "This pixel is lit up this much, this pixel's lit up that much." And then some black magic happens inside the perceptron, and it just says yes or no. That's the output.

0:16:29 MM: That's right.

0:16:30 SC: Like, "Yes, this is what I'm looking for, and no, this is not."

0:16:32 MM: That's right.

0:16:33 SC: And, what was the difference then between that and a neural network?

0:16:37 MM: So, two things; one is that perceptrons had no hidden layer. No hidden layer. The word hidden is a little strange here, but this is what people call it in the field. It just means that there's a layer of neurons between the inputs that where information propagates from the inputs to the hidden layer to the outputs. And the hidden layer is able to somehow create some kind of internal representation of the input. So, if it's a hand written character, if you're showing it, the input is just, say all the pixels, they're either black or white. Well, the hidden layer might be able to learn to represent something like, if you want to say represent the handwritten digits like one, zero through nine, you might be able to represent the idea of a loop or a circle. So that's intermediate representation.

0:17:32 SC: Yeah, it's not just what's going on in one pixel, it's some global holistic property.

0:17:36 MM: Right. And this seems to be what the brain does in some very...

0:17:39 SC: Absolutely. Yeah.

0:17:40 MM: Rough sense in the visual cortex. So, perceptrons did not have that hidden layer, they just had an input layer and an output layer. And the problem was that they... Because they weren't able to create those internal representations, they weren't able to learn more complex functions, more complex concepts. And, the problem with the hidden layer is that there was no learning algorithm that could go from the outputs and look at the errors made at the outputs, and apportion credit for, or blame for those errors, back through a hidden layer to the input.

0:18:29 SC: Through the different neurons. Yeah.

0:18:30 MM: Yeah. So it made the whole system much more complicated, and nobody had an algorithm for learning.

0:18:35 SC: So, the perceptron did have an algorithm for learning, so it would basically start with random inputs and outputs and then depending on how well it did...

0:18:43 MM: Well, random weights.

0:18:43 SC: Random weights, sorry that's what I mean. Yeah.

0:18:44 MM: Random weights. So... So each of the inputs had a weight.

0:18:46 SC: Right.

0:18:47 MM: Right.

0:18:48 SC: So... So, the perception carries in different amounts about what's coming in from each of its inputs...

0:18:52 MM: That's right.

0:18:53 SC: And then it... Those were set randomly, and then it would make a guess, that's what the answer is, and if it guessed right, it sort of... That was happy, and if it guessed wrong, it would adjust itself a little bit.

0:19:01 MM: It would adjust its weights. Right. And that's... That's what the brain does. Neurons have inputs, the connections between neurons are... Their weight can be changed. Their strength could be changed in the synapse. And that's what learning seems to consist of in the brain.

0:19:23 SC: So for the deep neural networks or just neural networks, in general, the trick was to teach the whole network how to learn in that sense?

0:19:29 MM: That's right. And not until the later part of the '70s, people come up with an algorithm called back propagation, which was able to train neural networks that had these intermediate or hidden layers.

0:19:45 SC: So, it was Minsky and Papert?

0:19:48 MM: They didn't come up with that algorithm.

0:19:49 SC: No, no. I'm sorry. Those are the authors who were skeptical about perceptrons and neural networks.

0:19:55 MM: That's right. And they also... When I went in and read some of the historical literature, I found that Minsky and Papert were very concerned about neural networks being sort of a competitor for their approach to AI, which is very different.

0:20:09 SC: Okay. What was theirs?

0:20:10 MM: Theirs was called, people call it symbolic AI, and it was much more based on humans, programming, and rules for behavior, and instead of learning and also having the rules be interpretable to humans.

0:20:29 SC: And the back propagation made neural networks much more successful than Minsky and Papert said they could be?

0:20:35 MM: Yeah. Minsky and Papert actually doubted that there could be a learning algorithm, but they were very quickly shown to be wrong, and they apologized later. [laughter]

0:20:42 SC: Okay, good. And is that basically the kind of learning algorithms we still use?

0:20:46 MM: Absolutely.

0:20:47 SC: Yeah.

0:20:47 MM: Almost completely. We use... People use back propagation...

0:20:51 SC: Okay.

0:20:52 MM: Which is based on calculus. [laughter]

0:20:56 SC: Good, calculus. Always a good thing, yeah.

0:21:00 MM: And it's very elegant, and it seems to work.

0:21:03 SC: Except that... Let's see. We've gone away a little bit from being inspired by the brain because the individual... Sorry, what do we call the individual nodes in the neural network, neurons?

0:21:15 MM: Some people call them neurons, some people call them units.

0:21:16 SC: Units. I wanna call neurons. I don't care. I don't care what the neuroscientists say.

0:21:20 MM: Yeah, yeah. Neuroscientists hate it when you call them neurons.

0:21:22 SC: They can adjust for the duration of this podcast. The individual neurons... We have a learning algorithm that propagates information backwards and forwards through the network. But in the real human brain, there's different modules. There's a visual cortex, there's the amygdala and whatever. And my impression, correct me if I'm wrong, is that neural networks start off pretty undifferentiated, pretty homogeneous. They're just like a whole bunch of neurons and they're gonna train themselves, and then wherever they go is wherever they end up.

0:21:56 MM: That's right. There's a lot of differences between neural networks in the brain. Most of the most successful neural networks people use today are very loosely based on the way the visual system works, at least as of 1950, the understanding... [laughter] And they're called convolutional neural networks. So, I don't know, some people in your audience probably have heard of these.

0:22:21 SC: So what does that mean? What does convolutional mean in this context?

0:22:24 MM: So the idea here is that, if you look in the visual system, each neuron, say, the lowest layers of the visual system. The visual system is organized into layers. The lowest layer, each neuron very roughly, this is a very rough approximation, has input from, say, the retina, and it's looking out of the visual scene, and it's sensitive to a particular part of the visual scene. A small part.

0:22:54 SC: Yeah.

0:22:55 MM: And what it does is very roughly equivalent to a mathematical operation called the convolution where it essentially multiplies its weights of its inputs times the input values in this small area and sums them up.

0:23:11 SC: So it's sensitive to some things and not to other things.

0:23:15 MM: Which makes it sensitive to some things. And, in particular, the neuroscientists Hubel and Wiesel, who discovered this sort of structure found that these neurons were sensitive to edges, which makes sense for vision. You'd like to know where the edges are.

0:23:32 SC: I think so, yes.

0:23:32 MM: And there's sense... Each neuron is sensitive to particular kinds of edges, like some are sensitive to vertical edges, some are sensitive to horizontal edges, some are sensitive to edges in between. So that's what a convolutional network does is, it has this idea of each neuron, having a small sort of receptive field, that is, it pays attention to a small part of the visual field, and it does these convolutions which are essentially detectors for things like edges. Nobody actually programs them to specifically detect edges. But if you actually train them on lots and lots of images, the lowest layer in a convolutional network will develop edge detectors just like the brain.

0:24:18 SC: Presumably, because that's a useful thing for computers to be able to recognize just like us.

0:24:22 MM: Yeah. Then there's a claim that they develop similar kinds of representations and even higher layers to what's seen in the brain, but that's a little bit harder to test and [0:24:32] ____.

0:24:32 SC: Well, this is a big part of the problem, or at least, again, in my medium-level understanding, we can get deep learning neural networks that are very good at certain tasks, but then if you ask why they are good at it, what is going on inside those hidden layers, it's kind of hard to suss out.

0:24:49 MM: Exactly. Neural networks are pretty complex systems. They have, especially the ones that are used in real world tasks today, have millions of weights in them, some even billions of weights, and it's really difficult to figure out what they're doing. It's not like the Marvin Minsky approach to AI where a human is building in the rules. I say, if you're driving in your car and you see a red stop light ahead, stop. Instead, that is somehow included in these billions of weights, in a way that's very hard to visualize or pinpoint.

0:25:34 SC: And it makes them, this is jumping ahead maybe a little bit, but it makes them easy to fool, right? There's a training set. We give them certain information. If you're just trying to recognize digits or recognize stop signs or whatever, and then it learns a certain thing and then if you know that it's good because of the certain training set, you can give it an image that we human beings would instantly recognize and they would get wrong.

0:25:56 MM: Yeah, they are very easy to fool. That's for sure. And it seems that a lot of machine learning systems, not just neural networks, are easy to fool. Big data somehow causes networks to focus on certain features that make it fairly easy to fool them, if you know what you're doing.

0:26:17 SC: I read recently that image recognition algorithms are much more sensitive to textures than human beings are, rather than edges. So, if you showed a picture of a cat with the skin texture of an elephant, it will certainly say it's an elephant.

0:26:30 MM: Right. So that's something that I think a lot of people don't really... It's not obvious when you're looking at these networks. They perform really well. Think of a face recognition neural network. You train it on millions of faces and now it's able to recognize faces, certain people. We humans look at it and say, "Oh, it's good at recognizing faces." But you don't actually know for sure that that's what it's recognizing. It may be that there's some aspect of the data, some, as you say, texture or something in the data that it's very sensitive to, that happens to be statistically associated with certain faces. And so, it's not necessarily learning to recognize the way we recognize, the way humans recognize. It may be something completely different that may not generalize well.

0:27:32 SC: Yeah, when the context changes, that correlation might completely go away.

0:27:36 MM: And that's something that people have found with these neural networks, is that, not only are we able to fool them, but even if we're not trying to fool them, certain small changes in the data that they're given that they're slightly different in certain ways from the input will cause them to fail. One recent experiment, they trained a neural network to recognize fire engines, fire trucks, right? Then they photoshopped images of fire trucks in weird positions in the image. Upside down, sideways, in the sky. And the network...

0:28:14 SC: Had no idea.

0:28:15 MM: Completely misclassified it, even though a human would be able...

0:28:18 SC: Yeah.

0:28:19 MM: Would recognize them. So then, when we say we've trained them to recognize fire trucks, it's not totally clear what we've actually trained them to recognize. That's a little bit of a difficulty in neural nets.

0:28:30 SC: Well, and this is one of the reasons why some of the most impressive successes of AI programs have been in very well-defined, finite situations like games, right? Like chess and go, and so forth.

0:28:40 MM: Yes.

0:28:40 SC: It's clear what the rules are. There's no equivalent of an upside down fire truck in a chess game.

0:28:45 MM: Right.

0:28:45 SC: And then, but it's also a reflection of the fact that the way the network is doing it is a very different way of thinking than human beings are doing it. Should we be very happy, very impressed or a little bit skeptical about the successes of these computers at winning in go or chess?

0:29:04 MM: I would say both, a little of both. It's very impressive. Chess and go are really hard games. People train for years to be good at these games, and now machines have surpassed them in all of these games, which is really impressive. Go has been a long-term grand challenge for AI. On the other hand, it's not totally clear that these same techniques that make them so good at go and chess are going to carry over to any real world, if you will...

0:29:39 SC: Yeah.

0:29:39 MM: Ability. And we really haven't seen much of that in...

0:29:44 SC: Well, I guess my contrary take is, I've never been impressed with the fact that chess and go are regimes in which computers can beat us. I would think that those are the most obvious places where computers should be able to beat us. The impressive thing to me is that human beings, with our very limited number crunching capabilities in our head, are pretty good at chess and go.

0:30:05 MM: Right.

0:30:05 SC: If computers become good at soccer and baseball, I'd be much more impressed.

0:30:09 MM: Well, yeah. But I think that not everyone saw it that way. A lot of people saw go and chess as the pinnacle of intellectual power if you would have to... And a lot of very smart people literally said that back in the '60s and '70s, that if a computer could play chess at the level of a Grand Master, AI would be solved.

[chuckle]

0:30:31 SC: It's just very backward. I mean, human beings are really bad at taking cube roots of numbers and computers are really good at that. And chess and go seem to me kind of like that. We're impressed by them in our fellow human beings, because they seem like things that computers should be good at. That's my view. I don't know. Maybe I'm on the wrong track, there.

0:30:50 MM: No, I think that's very insightful and interesting. But it's, in some ways, it's looking backwards 'cause we've already seen them.

0:31:00 SC: I know, I'm too late. I should have written a skeptical book about this years ago.

0:31:05 MM: So people in AI sometimes complain about this attitude, that once a computer can do it, we no longer say it requires intelligence.

0:31:14 SC: That's fair. We are moving into the real world, right? I did see some robots playing soccer. But things like self-driving cars and automated driving is much more something we're pushing on very hard. What is your view of how that field is advancing?

0:31:30 MM: So self-driving cars is prototypical field for AI. It's one where people think it's gonna be very easy, that we're 90% of the way there and they think it's gonna be solved very soon, and yet, every time they try to deploy it in the real world, the real world bites back.

[chuckle]

0:31:55 MM: The real world is different from simulation. It's different from all experimental techniques. It's... Self-driving cars turned out to be a lot harder than people thought just like a lot of things in AI. And the reason seems to be that there are so many different possible things that can happen. And I think this is true in most of life, not just driving. But most of the time, you're driving along in your... Say, you're on the highway and there's cars in front of you there's cars in the back of you, and nothing much is happening. But occasionally, something unexpected happens, like a fire engine turns on its siren and starts coming by. Or there is a tumble weed in the road. I spent a lot of time in New Mexico.

0:32:55 SC: Yeah. [laughter] And even though these events are unlikely, they're crucially important that we get them right, right?

0:33:00 MM: Yes. And one of the problems with self-driving cars, I've been told, nowadays, is that they perceive obstacles all the time, even when there's no obstacle or human wouldn't consider the thing an obstacle. And so, they put on the breaks quite a bit.

0:33:16 SC: You had the example of seeing a snow man on the side of the road.

0:33:19 MM: Right. So they don't know what to do. Say, there's a snow man on the side of the road, so that's unlikely, but could happen. A computer has no way of knowing that that thing is not alive, and is not gonna cross the street. It has no way of knowing if a dog is on a leash or not. It's gonna run out in front of traffic. It doesn't... There's a lot of things that humans can tell very clearly by their behavior, what they're gonna do next, but self-driving cars have a hard time predicting.

0:33:54 SC: And it seemed to me that this is at the heart of it, it's not just that these arenas are more complex than chess or go, but that it's a place where we humans have a competitive advantage. We have some picture of the world built into us that helps us answer some of these questions. Whereas, if you're just a big neural network trained on a bunch of test cases, you don't have that common sense that we humans come equipped with.

0:34:18 MM: Right. So common sense is a really important idea in AI. It's been talked about for years and it's kind of an umbrella term for all the stuff that humans can do without even thinking about it.

0:34:33 SC: Yeah, if I put the glass on the table, it will not fall through.

0:34:36 MM: Yeah, it's all the stuff we know that we either were born knowing or learned very early in life. And we have models of the world that give us common sense that we just have a vast amount of knowledge. Nobody knows how to model that in computers. There's a lot of effort being put into it.

0:35:00 SC: Well, you mentioned this thing I'd never heard of called Cyc, which is short for encyclopedia in this case. And I guess that was Douglas.

0:35:06 MM: Lenat. Lenat.

0:35:07 SC: Lenat?

0:35:08 MM: Yes.

0:35:08 SC: And an attempt to codify, a decades-long attempt to codify what human beings take for granted.

0:35:15 MM: Exactly.

0:35:16 SC: In just a list of propositions or how is it encoded?

0:35:18 MM: It's encoded in a list of propositions in a logic-based language and it's able to make deductions and deduce new facts.

0:35:27 SC: So what kinds of facts count as common sense to this program?

0:35:30 MM: Every living... Every human has a mother, every human has a father. A human cannot be in more than one place at a time. If you leave a coffee cup full of coffee out in a cold room, it will cool down. Any... Everything. You can't imagine listing all of the things that you know.

0:35:58 SC: And how long has this been going on, the attempt to make all these?

0:36:01 MM: Maybe 30 years.

0:36:03 SC: And there was... There is... The guy said, that they may be 5% of the way?

0:36:08 MM: Yeah.

0:36:09 SC: Do you believe that figure whatsoever? 5% of the way to understand common sense? [laughter]

0:36:13 MM: No, I have no idea how he got that. That seems crazy. I mean, there's no way you could possibly list all of the individual propositions of human knowledge.

0:36:24 SC: And does it even seem like the right thing to do? Maybe we've learned from certain artificial intelligence and just like... Like AlphaGo, which is with the best Go player, it eventually... They figured out that it was better just let a train itself rather than to learn from human beings. But maybe there's some advantage in this wider context of teaching it something about the world, rather than just a list of true things like something about Physics and something about the folk reality that we live in.

0:36:53 MM: Right. So humans have this knowledge that people call intuitive physics.

0:37:00 SC: Yeah.

0:37:01 MM: I know that if I drop a ball it's going to fall, and I know if it's made of a certain material, it's gonna bounce, if it's made of other material, it's not gonna bounce. I have all these things that I just know because either it's innate or I learned them when I was a baby. And there's a lot of theories about developmental psychology, about how human babies learn things and animals and so on. We also have some things that are even more basic than that. We have this idea of cause and effect, that certain things can cause other things. Neural networks don't necessarily have that.

0:37:43 SC: No, not at all.

0:37:43 MM: They have no notion of causality, but that seems to be really important. We have this notion, there are objects in the world, the world is divided into objects.

0:37:51 SC: They have some permanence although is not complete.

0:37:53 MM: They have some permanence and that they have certain properties and neural network doesn't know that, and it's not clear that it could learn that.

0:38:03 SC: So if some object disappears behind a barrier, it would eventually reappear something a human being would think but the neural network might have no idea.

0:38:08 MM: But even the notion of an object itself.

0:38:11 SC: Yeah.

0:38:11 MM: So one example was, in addition to AlphaGo the Google Deep Mind group did some work on Atari video games. And there was one game in which you take a joy stick and you move a paddle to hit a ball. This is all in software. Hit a ball to knock out bricks. It's called Breakout.

0:38:34 SC: Breakout. I know it well.

0:38:35 MM: Breakout. Yeah, it's a fun Atari game. So they taught... They used reinforcement learning just like in AlphaGo to teach the machine how to play Breakout. It only learned from pixels on the screen. It didn't have any Notion built into it.

0:38:50 SC: So it doesn't think, "Oh, I have a paddle. There's a ball, there's a wall. It just... "

0:38:53 MM: No. But it learned to play at a superhuman level. But then another group did an experiment where they took the paddle and they moved it up two pixels. Now the program could not play the game at all because it hadn't abstracted the notion of a paddle as an object. It's just a bunch of pixels. It was as if we would see the world and not see objects.

0:39:17 SC: But what's the lesson there? Is there a way that we can... Is there a more efficient way of teaching the computer how to think about the world to give it some common sense?

0:39:26 MM: We may have to build some things in. It may be that things are built into our brain by evolution.

0:39:31 SC: Sure. That would be the least surprising thing in the world really, right?

0:39:33 MM: Yeah. I know. But there's a thing, a debate in cognitive science, if you will, or AI for 100 years about innateness, [chuckle] what we learned versus what's built in. And no one really knows for sure, but there's a lot of evidence that there are sub-innate concepts that we are born with, they're just given to us and they bootstrap our learning, we can't do it without them. And so we may have to build some things into our AI programs. So this is just anathema to deep learning people, a lot of deep learning people, because deep learning, another machine learning, was put in opposition to the old-fashioned way of doing AI where everything was built in. And that turned out to be very brittle and not very adaptive. So people want to just focus on learning everything from data.

0:40:25 SC: I know that Judea Pearl, who's the doyenne of causality, gives AI researchers a hard time exactly for this reason, that cause and effect relationships are just not things that they program into the computers.

0:40:36 MM: And it's not clear you can learn that... Even the concept of cause and effect, from a bunch of data.

0:40:43 SC: Well, is there some intermediate where we don't teach the computers physics or intuition, but we do teach them some higher level concepts? We teach them metaphysics.

[chuckle]

0:40:53 MM: Yes. Yeah, that's right.

0:40:53 SC: We teach them objects and providence and cause and effect and they learn what... They could learn how many dimensions there are in space maybe just by experiencing things.

0:41:01 MM: Right. So there are some people who are working on that kind of thing, on building in some privatives and then having the systems say, "Put in virtual reality and try and learn physics."

0:41:14 SC: Oh good. How is that going? [chuckle]

0:41:17 MM: It's... There's some good demonstrations, but it's not very general. I think it's a very hard thing to do, and the virtual reality we have today is maybe not... I mean, it's... Again, virtual reality can be very complicated too. And so it's hard to learn.

0:41:37 SC: Do you think this is a promising way forward to try to teach some of these fundamental issues? No, what should I say? Fundamental metaphysics?

0:41:44 MM: Yeah, I think metaphysics is a perfect word for that.

0:41:46 SC: Yeah, it really is. Okay, good.

0:41:48 MM: Yeah, I think there is, and in fact, DARPA, the Defense Advanced Research Projects Agency, which funds a lot of AI, has a big push on what they call foundations of common sense. They're funding a lot of work on this.

0:42:00 SC: Ah really?

0:42:02 MM: And their goal is to basically have the groups simulate a baby from zero to 18 months. They have all the developmental milestones that they've gleaned from the psychology literature, and they want the artificial baby to essentially go through these developmental milestones. It doesn't have to be an actual robot baby, but some kind of simulation. And it's gonna be tested with the same kind of psychological experiments that people use on babies. So that's their idea. And it's gonna learn from videos and virtual reality. We'll see.

[chuckle]

0:42:37 SC: We'll see how that happens. Yes. That's right. But I don't wanna completely lose track of this thing you said about how this approach is getting pushback from the deep learning experts.

0:42:47 MM: Right. So there's many people in deep learning who feel that building things in is cheating.

0:42:54 SC: Is it just cheating or is it ineffective?

0:42:58 MM: I think they feel that it's in some sense both just as it was back in the good old-fashioned AI days, that it makes the system unflexible, that we don't know what to build in, just we can build in rules but those rules will be unflexible because we can't foresee every possibility and that things need to be learned by the system.

0:43:26 SC: Are those the same people? I guess I'm not very sympathetic to this, I have to say, 'cause I do think the common sense is secretly really, really useful and training on huge data sets won't necessarily get you there. I recently appeared on a podcast with Lex Fridman, who is an artificial intelligence guy. So I went as far as to say maybe programs that try to recognize the number three, the digits of the numerical system, would be aided if they understood the concept of three, and he thought that was terrible. He thought it was a very bad idea.

0:43:57 MM: Oh, that's an interesting idea. So there is a group who is working on this kind of thing. I'm thinking of, for instance, Joshua Tenenbaum at MIT and his group and some of his former students, are looking at that kind of thing, recognizing characters. And by understanding the characters more deeply than just the visual sort of presentation of the characters. And one of the things they wanna be able to do is to understand how a character was written in terms of the actual pen strokes and to be able to reproduce that and to learn from that so as to be able to generate the thing as opposed to just recognize it.

0:44:43 SC: Yeah. My worry is that if we teach computers this way, they will no longer be able to take cube roots or play chess very well or anything like that. They'll become too human-like.

0:44:53 MM: Well that may be an inevitable evolution. So in Doug Hofstadter's book Gödel, Escher, Bach... This was written in the '70s, so a long time ago, he actually has a set of 10 questions and speculations on AI, and one of his questions was will a smart computer be able to add fast? Which is exactly that question.

0:45:16 SC: Yeah, and what was his answer?

0:45:17 MM: His answer was no because exactly what you said.

0:45:23 MM: Our numbers, to us the concept of three is a full-fledged concept that has a lot of baggage.

0:45:28 SC: Yeah, a lot of baggage.

[chuckle]

0:45:30 MM: Exactly, a lot of baggage. And when we think about three, we don't just have some bit string representation, we have all kinds of associations with it, which maybe makes it hard for us to add things.

0:45:47 SC: I did read Gödel, Escher, Bach when I was a kid, so maybe this is why I have these infarctions.

0:45:50 MM: Yeah. No, it's a great question, and I think to me, I always found that one of the most surprising things I read in that book and... But thinking about it, I think it makes a lot of sense that maybe there are trade-offs, that we can't have human-like concepts and also get rid of all of our human slowness and the rationality and cognitive biases and all of what have you.

0:46:14 SC: I do think that as a practical matter, there shouldn't be a problem giving a computer a subroutine that can add and subtract and multiply and divide.

0:46:22 MM: We can give it a calculate.

0:46:22 SC: Yeah, exactly.

0:46:22 MM: Just like we... But it won't feel that the calculator is part of it.

0:46:26 SC: Right. That's right.

0:46:27 MM: It's not part of itself.

0:46:29 SC: Well, this is getting us toward... There's one thing to be generally intelligent in the sense that... I think general intelligent is not the right phrase. There's one thing for an AI to be able to work in the real world, like self-driving cars, there's another thing for it to be human-like in its intelligence. Are you someone who thinks that we are going to get there any day or the Google people in your introduction seemed very optimistic.

0:46:53 MM: Yes, and a lot of people are very optimistic. I'm not so optimistic because I think human level intelligence is a lot more complex than people realize, 'cause a lot of it's unconscious, we don't experience a lot of our intelligence consciously. And this is a problem that I think bleeds into people's view of AI. They think certain things are very easy and certain... And one of the things I quoted Marvin Minsky on is easy things are hard.

0:47:29 SC: It's a great quote, yeah.

0:47:30 MM: That things that we think are very easy like looking out in the world and describing what we see or recognizing your mother, or all these things turn to be turn out to be very difficult in general for computers. Whereas the things we feel are very hard, like playing Go they turn out to be able to do well.

0:47:56 SC: Well, we do have chat bots, we do have Siri, things like that. We can mimic at least the rudimentaries of human speech interaction.

0:48:04 MM: Right.

0:48:06 SC: The rudiments.

0:48:07 MM: The rudiments. So a big question is, is that on the right path to actually full-blown human-like conversation.

0:48:16 SC: The quickest way to do something 50% well, might not be the right way to eventually doing it 99% well.

0:48:21 MM: That's right. So Hubert Dreyfus, a philosopher who was a well-known AI skeptic had this analogy that you could climb the tallest tree around and say you were closer to the moon, but really, best way to get to the moon is to get down from that tree and get on a rocket ship.

0:48:39 SC: Yeah. That seems very, very... A good analogy for many problems in AI. One issue is embodiment. I think a few other times in the podcast when I've talked, not to computer scientists but to neuroscientists the point is made that just human thinking is so enormously affected by the fact that the brain is in the nervous system, is in the body, is in the environment and to date, a lot of AI is still on a computer screen, not in a robot.

0:49:05 MM: Right. I think AI people are secretly Cartesian in their dualism. They wouldn't admit it, but they don't believe in... They believe that intelligence is all in the brain. There are a lot of people who are doing what they call embodied AI. Mostly having to do with the robots, where actual sort of intelligence is spread throughout... You have to have the right kind of body. I tend to agree with the embodied people, that we can't have human-like intelligence without a body. I won't go so far as to say we can't have intelligence 'cause I don't really know what that term could possibly entail, but human-like intelligence is very much rooted in our bodies. That's how we think about concepts. We think in terms of metaphors, having to do with the body and physical space and time. We understand abstract concepts and analogies very physically. So I think that's just going to show more and more as we try and push AI further and further.

0:50:22 SC: Well, one thing that AI is good at, maybe not AI but computers are good at these days are deep fakes. Making images or even videos that look like another person is actually saying them. So I'm sure people have imagined this, but won't it happen before too long, that we wed a chat-bot or something like Siri to a deep fake and can actually mimic not only a video of someone saying something but an interactive conversation with an actual person, and you wouldn't know, you'd think you're Face-Timing somebody but you're just Face Timing an AI.

0:50:58 MM: That definitely could happen, but I don't think right now Siri and its cousins are good enough to fake it.

0:51:06 SC: Not right now.

0:51:08 MM: So that's a really good question.

0:51:09 SC: There's always the question of how fast these improvements are.

0:51:12 MM: How will more data make them better? Turns out that language understanding is much more difficult than vision, which is interesting maybe because at least the kind of vision... Vision in terms of object recognition and so on, that language understanding involves common sense. It requires a huge amount of knowledge. And AI is much less advanced in language than it is in vision I would say.

0:51:39 SC: This is why the Turing Test is hard to pass, right?

0:51:42 MM: Well, it depends on the details. So there have been chatbots that have "passed the Turing test" with certain kinds of judges. It's really hard to... A lot of people are very willing to anthropomorphise computers and to believe that they're actually human-like.

0:52:04 SC: I think that's what it is. Whenever I see these reports of a computer winning the Turing test, passing the Turing test, and then you see some of the actual conversations, I'm just shaking my head and thinking like, "How in the world can you think that was a person?"

0:52:15 MM: Yeah, I mean if you had to interact with Siri for 20 minutes talking to it, it would never fool you into thinking it was a person.

0:52:24 SC: I do wonder whether or not we also though, overestimate how complicated people are. Is it conceivable that many people actually have a relatively limited set of things they would say or do in any certain set of circumstances and that might be mimic-able in the foreseeable future?

0:52:45 MM: It depends. I guess, it depends on the idea of limited, how limited, but sure. I think these chatbots can carry on a conversation. A lot of times they have what we might call cheap tricks. They change the subject a lot, they answer a question with a question, they are able to deal with when they can't.

0:53:12 SC: Sticky situation, yeah.

0:53:13 MM: Yeah. But sure, there's one chatbot I know that learns by... It just kind of repeats different things that it's heard from other people and it sounds kind of game-like.

0:53:26 SC: Well, that's what I'm thinking. I, like many people who are not experts in this field, I'm heavily influenced by science fiction depictions of these things.

0:53:32 MM: Oh right.

0:53:34 SC: And Iain Banks, I don't know if you've ever read his science fiction books.

0:53:36 MM: No.

0:53:36 SC: But one of the concepts that he has is that people can be mimicked by computers at some 90-some percent level. You can basically download not a copy of your brain, but a much more compact representation of what you would usually say in typical circumstances, and that can outlive you when you die, right? And so it's like the portraits in Harry Potter where there's sort of a slim faint reflection of who you really were, but something that people could still interact with after you've gone. That doesn't seem crazily unreachable to me.

0:54:12 MM: Yeah, I think you would never actually be fooled by it.

0:54:17 SC: Yeah, you wouldn't be food but you might be able to say some things, learn some things, be amused at the very least.

0:54:24 MM: Yeah, I'm sure a lot of the things that I say every day are very stereotypical. Whenever I'm sending an email and I get the little suggestion from Google Mail about what I'm about to say, it's often right.

[chuckle]

0:54:36 SC: Pretty good. Yeah, that's right. That's right.

0:54:38 MM: And which means that I'm just very predictable and boring in a lot of ways, but I like to think that maybe that 10%...

0:54:45 SC: It matters.

0:54:45 MM: It doesn't get, actually matters a lot.

0:54:49 SC: But we already are in a world where holograms are doing world tours. Holograms of dead musicians are on tour now. And maybe given little bit of interactivity, you could go a long way. Doesn't need to carry on a conversation, doesn't need to be able to write a song, but it could respond to the audience or to the musicians.

0:55:06 MM: Yeah, I think that kind of thing could happen. And I do think a lot of cheap tricks can be very effective if you like.

0:55:14 SC: Right.

0:55:14 MM: The question is, is it cheap tricks, all the way down?

0:55:17 SC: Yeah.

0:55:17 MM: And I hope not. I don't think so, but it could be, I could be wrong.

0:55:23 SC: Well, and the cheap tricks... It sort of leans both ways. One of the things you talk about in your book are the ethics, the values that we are gonna attribute to AIs, the fact that an algorithm can be biased, I think, is one that some people have difficulty wrapping their minds around.

0:55:38 MM: Right. Algorithms learn from data, so if data is biased, the algorithms are biased. They just pick it up from the data. We have algorithms that are... Face recognition algorithms that are much better on white males, because they've been trained on data sets that are majority white males. They're trained on images that are uploaded to the internet and those tend to be well-known people statistically, who tend to be white males.

0:56:06 SC: Well-known to us yeah. [0:56:07] ____ to be well-known.

0:56:09 MM: And so people in AI have shown that these systems which people are using commercially are much more likely to make errors on say African-Americans. There was a really scary headline I saw that said self-driving cars are much more likely to hit black pedestrians than white pedestrians.

0:56:28 SC: Wow. I did not see that one. But when you say it, yeah.

0:56:30 MM: You would think it. Yeah. And a lot of the language systems have bias in their language that they... I saw one system that... Question, answering system that would answer questions about photos. And if you show it a photo of somebody cooking in a kitchen it's much more likely to think that that's a woman than a man because it'd see more women cooking.

0:56:54 SC: Yeah, It's not because women are intrinsically the ones who cook, but if you train them to think that. That's what they're gonna think.

0:57:00 MM: And these things can actually matter in real life, because they can... If they're used... Say, these biases are kind of hidden under the surface, if they're used in the real world for these systems making decisions and the decisions are not very transparent how they're being made the biases can kind of creep into them, so there's a lot of concern about that.

0:57:21 SC: And what is... I don't know, do you have a proposal for what to do about this or how we should be on our guard?

0:57:26 MM: Oh, wow, I think we need to have developed some tests of systems bias. We have people who have had these implicit bias tests they give to other people, and those are very revealing. You could also find such tests to give to AI systems.

0:57:46 SC: I see, so we're gonna start giving psychological tests to our computers.

0:57:49 MM: Yeah, absolutely, and we have to be more concerned about how we form data sets, how representative they are.

0:58:00 SC: So is the ethics not only what the AI does but of how we teach it.

0:58:04 MM: Exactly, yeah, and then there's people who are trying to find ways to "de-bias AI systems" with limited success so far. If you de-bias them too much, they start to make a lot of mistakes. It hurts their performance.

0:58:21 SC: Right.

0:58:23 MM: So it's an open research area, but important.

0:58:26 SC: And do you have worries about super-intelligent AI taking over the world?

0:58:30 MM: That's the least of my worries right now. I have a lot of worries. But that's the least of them. I'm a lot more worried about, say, these deep fakes for example, and just general fake media. And I'm worried about most like the issues of surveillance and privacy. I'm worried about the companies deploying AI systems that aren't ready, that are still... That don't have enough sort of quality assurance.

0:59:01 SC: 'cause people use AI for things like looking at resumes to pick job candidates, right?

0:59:05 MM: Yeah.

0:59:06 SC: Or sentencing recommendations in the justice system.

0:59:08 MM: Yeah. They'll use AI for all kinds of things, deciding if you're gonna get a loan, your bank is using AI and deciding whether you qualify for food stamps. More and more people are using algorithms to make decisions for us. And it worries me. But super-intelligent AI seems to me to be very far away.

0:59:34 SC: What would it mean? What exactly does superintelligence mean?

0:59:37 MM: Well, that's a thing, I'm not even sure. I'm not even sure it's possible, 'cause intelligence isn't well defined, it's one of those terms that we throw around all the time, but it means so many different things. So intelligence is very... If you were talking about sort of animal intelligence, it's very ecologically specific. It's specific to certain niches. Animals, including humans, are adapted for certain kinds of things. They do certain kinds of things well, they don't do certain kinds of things well, because it helped them to survive. And so when you talk about intelligence as being this sort of monolithic thing, oh, we're gonna have super intelligent machines, it's hard to know exactly what that means, right? If something could be sort of better adapted. Maybe these machines could understand... I was listening to one of your previous podcasts about how hard it is to understand the foundations of quantum mechanics. Maybe humans just don't have the right kind of brain for that. Maybe you need a machine to do that to figure that out because they have a different sort of ecological niche of understanding but would we call that a super intelligent machine? I don't know.

1:00:49 SC: Yeah, I think this is a very good question. I just last week was at a conference and heard Max Tegmark, give a report on stuff he's been doing with his students on AI physicists. So, theoretical physicists modeled by AIs. But all they're really doing is very, very fancy curve fitting. They see a bunch of data points, and they find the law of physics that best fits those data points. But asking what is the ontology of reality that would best explain quantum mechanics or something like that, is a little bit harder.

1:01:18 MM: So I'm being very speculative because I don't think any machine now has anything like the concepts that we have that would allow it to do science in any way, any meaningful way. But if you have to ask is super-intelligent AI even possible. You have to start thinking about what you mean by that and what we mean by intelligence, and nobody really has a good handle on what we mean.

1:01:42 SC: Well, I think the impression I got from your book is AI has become very good at very certain kinds of things, the kinds of things that are naturally adapted to brute force and deep learning, right? And it's a whole other set of kind of things that really are especially good... That humans are especially good at and the commonsensical things, maybe even the creative things. The creative things I'm actually less sure about than the commonsensical things. What are the lessons that this teaches us for how we should think about AI and how we should do research on it?

1:02:14 MM: Okay, that's a hard question. I don't know if AI is intrinsically limited. AI of course isn't just one thing, either.

1:02:23 SC: Sure.

1:02:24 MM: I don't know if machines let's say are intrinsically limited that there's things that machines are good at, and there's things that we're good at 'cause I think of us as kind of fancy machines.

1:02:34 SC: Fair enough. I'm on board with you there.

1:02:39 MM: So I'm not sure there's any intrinsic limitations, but we could talk about the current approaches to AI, this sort of data-driven approach. There's probably certain things that it's good at and certain things that it's not good at. We've seen that. So for me, there could be two lessons for research. One would be to try and understand better what kinds of things that today's AI are good at and how to maybe make it more reliable, or less vulnerable to fooling them... Fooling it, more of trustworthy and so on. But then there's the research into what AI is not good at to say, like, What is it about us that current machines can't do, and there's a lot of things, that we can do very well? One of them is forming abstract concepts, and that's something I'm particularly interested in. And so all of my research now is on more on that side. It's trying to say, what can't these machines do and why not? What are they lacking? And the reason I'm investigating this, is not to try and build Smarter Machines per se, but more because I wanna understand what intelligence is, and how maybe we do it or how it could be done more broadly. So I think of it as more of a scientific than engineering approach.

1:04:03 SC: So give us a hint. Well, how do we form an abstract concept? Is this something that you've learned something about in the research? Or why is it hard?

1:04:12 MM: Why is it hard? Well, okay, that's interesting. So here's an example: Think of the concept of sameness. Like two things being the same. That's an abstract concept, right? It's very abstract and it's something that deep neural networks can't do even in the simplest form. They can't recognize same... If you give them an example of something in which two things are the same versus two things are different in general, it can't do that task. Whereas we humans can, very easily.

1:04:48 SC: Well, I would say, we kind of can... If you ask people what they mean by same-ness, then you're gonna quickly get into a philosophical argument.

1:04:55 MM: Right, right, and a friend of mine actually wrote a book called the subtlety of sameness.

1:05:00 SC: There you go.

1:05:01 MM: Which is one of my favorite book titles ever. And it is extremely subtle, but we seem to be very good at it and it's kind of the root of all analogy. Analogy is a really important part of thinking and that's essentially seeing two situations as being two rather different situations as being intrinsically the same. I look at a particular political situation and I say... For instance, think back in history, we called the Iran-Contra deal Irangate that was making an analogy with Watergate, right? Very different situations, but somehow we saw them as the same kind of thing. It's a cover-up. It's a government cover-up, right? So that's a very subtle kind of sameness. And it's sort of underlying a lot of the thinking that we do in terms of analogy.

1:06:01 SC: Well, you have these wonderful examples, and maybe they're from Douglas Hofstadter. But how human beings think by asking, "If you're told that ABC gets converted to ABD, then what should PQRS be converted to?" Right?

1:06:18 MM: Right. So that was... That letter string analogies was a subject of my PhD dissertation, actually.

1:06:25 SC: There you go.

1:06:27 MM: So I built a program that could make those kinds of analogies in what we claimed was a human-like way.

1:06:36 SC: Because there's no right way, right?

1:06:38 MM: Yeah.

1:06:38 SC: So give some of the possible answers to this question. What should PQRS be converted to if ABC is converted to ABD?

1:06:44 MM: So ABC is converted to ABD. What should PQRS go to? Well, most people would say, "PQRT," because they see ABC changed to ABD, the right-most letter replaced by its successor. Okay. But you could also say, "PQRD." Replace the right-most letter by a D, or you could say, "PQRS." There's no C's or D's in PQRS, so don't change anything.

1:07:14 SC: If the rule was change every C to a D.

1:07:17 MM: Change every C to a D, right, or another answer could be ABD. Change any string to ABD. You can just go on and on and on. But people never... People are very good at being sort of abstract in the right way. Right being, I guess, defined to what people do.

1:07:39 SC: Well, in a way. Yeah, that's right. There's a universal way that people would do.

1:07:42 MM: In a way, yeah. Yeah. Humans are very consistent. Right. So the idea of that whole project was not to solve letter string analogy problems, but more to model analogy in general, and to say, that this little micro world of letter strings captured some of our analogical abilities in general.

1:08:02 SC: I think this is a deep thing, because there are an infinite number of rules that would always explain some situation. This is the Duhem-Quine thesis in philosophy of science. Have you ever heard of that?

1:08:13 MM: Yeah.

1:08:14 SC: Any set of data can be fit by an infinite number of theories. And I had a math professor when I was an undergraduate who hated the SAT and GRE series test. When you're given a sequence of numbers and you're told to guess the next one, and he goes, "Mathematically, it could be anything. I could invent any series of numbers that could be fit by anything next." But I think that the point was... My response to him was that they weren't actually math questions, they were science questions. They were looking for patterns in the world, and not all patterns are created equal. So in some sense, either because it's we humans or the world we live in with its actual rules acts in certain ways, and therefore it makes more sense to us that certain patterns are the right ones rather than the wrong ones. But it comes down to exactly this common sense, right, there's this way of dealing with the world that computers are not very good at.

1:09:04 MM: Right.

1:09:05 SC: And what did you learn by doing this PhD thesis about analogies?

1:09:10 MM: Oh, that there... They can be very subtle, and that... What did I learn? Well, we developed... Doug Hofstadter and I developed an architecture, sort of a computer architecture, for solving them that I think is more widely applicable. So I'm working on how does this apply, how can we apply these ideas and more broadly. And I think that that's an area of research that hasn't been looked at very much in AI, the whole area of analogy, and sort of common sense analogy.

1:09:43 SC: Do you see... Is there a movement in AI? Not just a few people, but is there a growing feeling that deep learning is a kind of thing, but we need more common sense, we need more folk intuition, more metaphysics, in our computers?

1:10:00 MM: A lot of people say that, not everybody.

1:10:03 SC: Okay.

1:10:03 MM: Yeah. I was kind of surprised. Recently, the Turing Award which is the highest award in computer science, was given to three deep learning people. And two of them, Geoffrey Hinton and Yann LeCun, both gave a speech at one of the big AI conferences... Or a conference. Yeah. And Hinton, particularly, was incredibly optimistic about deep learning, whereas LeCun was saying, "No. We need something different."

1:10:35 SC: Is there any chance that we could learn more about deep learning, or make it better just by asking it how it's thinking [laughter]? Are these demanding that when the algorithms reaches a certain conclusion, it can explain itself?

1:10:50 MM: Some people are trying to do that, but it's hard since, first of all, to explain itself, it has to have concepts 'cause it has to have a concept of what it did. It can't just say, "Well, I multiplied these 4 billion weights by these inputs, and then got this answer." Explaining is kind of relative to the entity you're explaining to.

1:11:15 SC: Yeah.

1:11:17 MM: And so if it wants to explain itself to humans, it has to have some kind of theory of mind of what counts as an explanation to a human.

1:11:24 SC: So I wanna be the deep learning proponent instead of skeptic for the purposes of conversation. Maybe it could learn that through deep learning, maybe that's... If we insist of one of its success criteria is, it can explain itself, then maybe it will be forced to come up with concepts and fit them together.

1:11:44 MM: Interesting.

1:11:45 SC: Easy for me to say.

1:11:46 MM: Yeah.

[laughter]

1:11:48 MM: Yeah, you'd have to figure out how kind of what language it would explain itself in and... I, you know, that sounds hard to me, but I think it's...

1:11:58 SC: I'm sure it's hard.

1:12:00 MM: It is important. There's this whole area called meta-cognition that people look at and it's the idea that we think about our own thinking, we're able to explain why we did things not always correctly. You sometimes rationalize why we did something or why we made a decision. But meta-cognition is really important part of intelligence, and it's something that people in AI, kind of, some areas of AI have kind of looked at, but nobody has any good ideas about how to do it.

1:12:30 SC: I mean maybe what we're doing is the equivalent of giving our deep learning networks multiple-choice tests, when we really should be asking them to show their work.

1:12:38 MM: Yes.

[laughter]

1:12:41 MM: Yes exactly.

1:12:43 SC: Alright, just to close up, then let's let her hair down and it's the end of the podcast and prognosticate about the future a little bit, I know that it's always very hard and I promise not to hold any bad prediction you make against you 50 years from now but... Just as one data point, what do you think the landscape will look like 50 years from now in terms of AI in terms of how general-purpose it will be, how much common sense it will have, how close it will come to being humanly intelligent?

1:13:12 MM: 50 years from now, wow.

1:13:14 SC: You can change it to another time, but I think 50 years is good because on the one hand, we'll probably be dead.

1:13:19 MM: Yeah, will the world last that long?

1:13:21 SC: On the other hand, like you can... Maybe you can accurately guess 10 years from now, but no one can guess accurately 50 years from now, so.

1:13:27 MM: Yeah. Yeah. Well, I can imagine that we would have much better chatbots that can do... The deep fake stuff would be incredibly good which is terrifying.

1:13:43 SC: Yeah. So there's already... Wasn't there recently this demonstration of a chat... Basically what a chatbot was, basically something that was a chatbot called up a restaurant and made reservations and it really fooled the person on the other end had no idea he was talking to a robot.

1:14:00 MM: Right. And that... A lot of conversation about should it be required to identify itself as a robot? And I think that kind of... There'll be a lot of legal frameworks built up over the next 50 years about AI and that kind of thing. I'm a little scared that we might live in a complete surveillance state, 50 years, that everything will be known about everybody.

1:14:26 SC: Yeah.

1:14:27 MM: And all of that data will be used to... Who knows what? Target ads to us.

1:14:33 SC: It seems basically inevitable to me, but...

1:14:35 MM: Have you seen the movie Minority Report?

1:14:37 SC: I have. Yes.

1:14:37 MM: Where... Yeah, that's like a real dystopia where it identifies you, it does an iris scan, and figures out who you are and then can... Constantly bombarding ads at you directed completely towards you or individually. That's horrifying. But if you look at current trends, that's almost where we're heading.

1:14:58 SC: Well, yeah, and I think even much worse than targeting ads like what if we become really good at deciding who will default on a loan and therefore, we don't give them a loan even if they haven't defaulted yet, right?

1:15:09 MM: Right.

1:15:10 SC: In some sense...

1:15:10 MM: And the whole Minority Report plot where we decide if somebody's gonna commit a crime.

1:15:14 SC: Yeah. They're trying to do that. That's perfectly legal for a mortgage company to try to decide the probability you will default on your loan, right?

1:15:21 MM: Yes.

1:15:21 SC: But if we become infinitely good at it, or at least we think we're infinitely good at it, then it's a different thing than what is the difference between that and saying, "Well you'll probably commit a burglary.

1:15:30 MM: Right. I think also because of co-evolution with these technologies that a lot of people will be working on how to fool them, how to thwart them, how to avoid this kind of thing. And there will be a big kind of co-evolutionary arms race between the technology and the people who don't wanna be controlled by it.

1:15:58 SC: Presumably, there's some good things. Maybe when I call to change my flight reservations I'll get a much more useful computer interface than I do right now.

[chuckle]

1:16:07 MM: But audio-visual won't be solved. Like being able to get your slides to show up on the computer.

1:16:14 SC: Yeah, no, no, no, that's true but I guess... Now, do you think that will be... You said audiovisual, and my brain went to the wrong place, will it be video calls rather than audio calls. This is always...

1:16:26 MM: You know we already have that.

1:16:27 SC: We have it, but people don't use it that much, right?

1:16:29 MM: No. Yeah, probably people will use it more.

1:16:32 SC: Yeah, weirdly we use text messaging now more than phone calls so that we've gone downward in the amount of information sent.

1:16:39 MM: Yeah.

1:16:40 SC: The kids today, they like to text on purpose.

1:16:41 MM: Yeah, yeah. We might... It's possible that people won't wanna interact at all.

[laughter]

1:16:51 SC: Well, again, I'm trying to look at the sunny side of this thing. We can imagine that if we do have the equivalent of deep fake chatbots, they will be useful companions for people, for lonely people, for elderly people, or whatever. There's a sort of robots who are already doing this, but I think they'll become better at that.

1:17:08 MM: Yeah, that's right, that's right. And the effects of that are completely unknown.

1:17:13 SC: Yeah.

1:17:14 MM: I find that a little scary.

1:17:17 SC: Are you mostly scary or are you mostly excited about the robot AI future?

1:17:24 MM: It varies. I'm very excited about... I tend to get more excited when things don't work.

1:17:31 SC: You're a scientist, of course, sure.

1:17:33 MM: Yeah. 'Cause I wanna understand why they don't work. And that... So I don't know if that makes me kind of a pessimist. I think there's a lot of things... Sort of academic... This is a whole different discussion, but sort of the academic research in AI is badly broken in many ways in that people tend to focus on particular benchmark data sets and see who gets the best performance on them and the state-of-art in the scientific approach to AI isn't really progressing very much, I think. But...

1:18:18 SC: Sorry, what's the relationship between those two facts?

1:18:21 MM: Oh, well, suppose that you want to get your paper published in the top conferences in AI, the best thing to do is to take a benchmark dataset that a lot of other people have worked on, and to do better than anybody else, not to explain anything or to understand why your system did better.

1:18:40 SC: So you're saying that we're becoming like the deep learning networks?

1:18:43 MM: We are, we're being optimized according to these benchmarks. That's right. We get rewards in the form of grant money and.

1:18:51 SC: Yeah.

1:18:52 MM: Recognition.

1:18:52 SC: Hasn't it always been like that, is that... That's really not a new thing.

1:18:55 MM: I don't know, I don't know if... Maybe a little bit.

1:18:58 SC: We're always optimized for our lock of reward system, I think.

1:19:00 MM: Yeah, but it's not always the best for, you know, progression of science.

1:19:05 SC: It's true, it's hard to... Like science does progress pretty well despite the fact that there are so many obvious flaws in the system, that it's hard to know where to start listing them.

1:19:14 MM: Yeah. [chuckle]

1:19:17 SC: But I like your philosophy of being excited and happy when things break and go wrong. I will count that as an optimistic stopping place because I'm sure things will continue to break and go wrong.

1:19:26 MM: Absolutely.

1:19:27 SC: Alright Melanie Mitchell, thanks so much for being on the podcast.

1:19:29 MM: Thanks for having me.