343 | Tom Griffiths on the Laws of Thought

0:00:00.3 Sean Carroll: Hello everyone, and welcome to the Mindscape podcast. I'm your host, Sean Carroll. I've always thought that one of the interesting aspects of modern approaches to AI, large language models, other connectionist things, is that very often, or at least in their natural state, an LLM is not good at arithmetic. It's not good at adding numbers together. You can augment the program so that they're very good. You can give, basically an LLM access to a calculator. It's exactly like human beings. They're not very good at arithmetic in some sense, but if you have a calculator, they can do it. But in their natural state, LLMs make mistakes about simple problems, adding medium-sized numbers together. Other kinds of number-based problems, they're not very good at. Counting the number of R's in the word strawberry. Random numbers. If you asked an LLM to generate a million random integers between 0 and 100 and then made a plot of the frequencies, so it would not look uniform. Of course, these are all things that human beings are also intrinsically not very good at, but part of you thinks, "Come on, it's a computer, it should be able to do simple arithmetic problems."

0:01:11.0 SC: And of course, the answer is there's no real mystery here. The computer on which the LLM is running has no problem doing arithmetic, but you're not talking to the computer, you're talking to a program. And the program might not be set up to do those kinds of things. And again, it's exactly like humans. It's almost like you tried really hard to make a program that sounded human, and in the course of doing that, it lost the ability to do arithmetic, which is kind of interesting when you think about it. But it's also a reminder that when you say thought or thinking, you're not really referring to a single thing. The ability to add numbers together and the ability to carry on a conversation, those are two very different abilities, and you might optimize for one over the other in building a program or evolving an organism through natural selection. Nevertheless, we do sort of aim at a sort of standard set of standards for thinking correctly, right? We want to get the right answer when we add numbers together. We want to logic our way through puzzles that we are given. We want to reach rational, reasonable conclusions.

0:02:19.7 SC: So how do you sort of fit together, on the one hand, the pristine rules of logic and reasoning to which we aspire as thinking reasonable creatures, and on the other hand, the reality of our minds and our brains and our embodied intelligence, which has, number one, a whole bunch of different things that it was selected for over the course of biological time, and number two, all sorts of constraints in terms of energy and fuel and time and things like that. If you had a brain, a human kind of brain that was able to do arbitrarily good arithmetic, that might make it worse at other things that were more important for survival. So the idea, this is a very hard problem for cognitive scientists, the idea of coming up with the laws of thought, the laws that an ideally rational creature working under certain constraints would follow. Those constraints include not only the fact that there's finite resources, but also things like you might not be certain about the facts, right? You might have to be a kind of Bayesian or nearly Bayesian reasoner who has probabilities for certain things being true and then learn to update them, and so forth.

0:03:33.6 SC: So the quest for these laws of thought is what we're talking about today with Tom Griffiths, who's a cognitive scientist. He has a new book coming out called, guess what, The Laws of Thought: The Quest for a Mathematical Theory of the Mind. As I learned in the conversation, he has another book coming out also right now, which is sort of the technical version of this in some sense, or at least a companion to it. So The Laws of Thought is meant for everybody. The other book is with co-authors Falk Lieder and Frederick Callaway, and it's called, 'The Rational Use of Cognitive Resources: A New Approach to Understanding Irrational Behavior, Modeling Human Cognition.' Part of what you learn by thinking about these things is that certain ways in which human beings act irrationally, if you compare them to the perfect laws of logic that you might aspire to, actually have good reasons for them, right? There are reasons why we have certain inclinations, certain biases, and so forth. What does all that mean for the world of programming artificial intelligence? Should we try to make the AIs think just like human beings? Are there shortcuts? Are there better things that we can do? After all, the AIs also have constraints, but they're not the same constraints that human beings have. So figuring out why cognition is the way it is, and therefore how to implement it better in different kinds of systems is, it's going to be a growth area in the near term. Let's just put it that way. So, let's go.

[music]

0:05:14.3 SC: Tom Griffiths, welcome to the Mindscape podcast.

0:05:16.3 Tom Griffiths: Yeah. It's great to be here.

0:05:17.5 SC: You know, I'm a physicist, and I often sort of teasingly say that physics is good for people with short attention spans, no real ability to keep a lot of complexity in their minds all at once. And at the end of doing physics, you come up with the laws of physics. As a cognitive scientist, as someone studying literally the most complex thing that we know about in the universe, is there any hope that we should even talk about something like, 'Laws of Thought,' the title of your new book?

0:05:48.4 TG: Yeah, I mean, I think there is. So one interesting fact is that the people who set out with starting that enterprise of mathematical physics had in mind doing the same thing for understanding how thought works, right? So, the very earliest philosophers and sort of scientists, and sort of in the inception of science itself, were really thinking about mathematics as a tool for understanding both the external world and the internal world. And we see that in Descartes and Leibniz, and many of these people sort of talking about that idea. And so I think it's possible for us to end up with something that looks like a set of laws, as long as we think carefully about what it is that we're trying to characterize. So, we want to understand how intelligence works. And there are many different ways you could think about that. You could think about it abstractly in terms of what are the general principles that would govern intelligence, and concretely, in terms of how is it that things like brains work in order to produce intelligence? And at those different levels, I think there are things that we can point to that are generalizations that maybe have the right character to be things that we could call laws.

0:07:05.8 SC: I'm not even sure if this is a question, but it is fascinating to me, if you go all the way back to like Aristotle, et cetera, that Aristotle talked a lot about the world, the natural world, and the discussion of what we would call the physical world and the biological world. There were distinctions drawn, but it was kind of a continuum, right? Like, he kind of tried to use the same concepts in both cases. I don't know at what point that ceased to be a popular strategy.

0:07:35.1 TG: Yeah, and I mean, it's worth pointing out, he was also in many ways our first cognitive scientist, in terms of trying to come up with the laws that characterized. In his case, it's not clear whether he was focused on argument or thought, but doing some of the first work in logic that really provided the foundations for modern approaches to mathematical logic.

0:07:58.4 SC: Well, which brings up the question, when we talk about logic, if we just use the phrase laws of thought, is this meant to refer to the normative ways that you should be thinking, the right way of thinking, or is it just a descriptive phrase of, like, this is how thinking actually gets done?

0:08:16.5 TG: Yeah, so I think that gets us back to this idea of what levels we're trying to understand thought at, right? So, in cognitive science, following the work of David Marr, we talk about there being different levels of analysis that we can apply to information-processing systems. And so the most abstract of those is what Marr called the computational level, which is trying to understand what it is that a system is doing in terms of what problem it's solving, and then what the ideal solution to that problem looks like. And so that's really trying to ask a question about the function of the system and the goals of the system, and then what's an ideal solution to execute that particular function. And then below that, we have what he called the level of representation and algorithm, or the algorithmic level, which is more about what are the actual cognitive processes that you could engage in in order to produce something, which is maybe an approximation to that ideal solution. And then below that is the level of implementation, which is how is that realized in a physical system, right? And so, for humans, that's in brains, for computers, it's in silicon, and so on.

0:09:24.5 TG: And so, when we talk about identifying laws of thought, I think the most natural level to think about that is that most abstract computational level. And there are things like logic and probability theory stand out as the principles that we can use for saying how it is that you should be solving the kinds of problems that minds have to solve.

0:09:48.0 SC: And I want to sort of back up and talk about Leibniz and logic and things like that, but just to help the listeners with the roadmap of where we're going, what is your shortest answer to the question, so what are the laws of thought?

0:10:06.6 TG: So, it's a great question, and I think it's something where, when I started out writing the book, I wrote a sort of introduction that said what I normally say in my cognitive science classes, which is, we've done a lot of work here trying to understand how minds work. And in many ways, what cognitive scientists have done is kind of figure out better ways of asking the questions that we want to ask without necessarily giving us answers to those questions. But then having spent a few years working on the book and writing it, and having things in the external world having changed a little bit as well, by the time I got to the end of it, I ended up feeling like, actually, we've done a pretty good job of characterizing what some of those sort of abstract laws might look like. So, in particular, at that computational level, I think it's pretty clear that the things that we should be thinking about are things like logic and probability theory, and then some additional things that I don't really talk about in detail in the book, but which are more about, how that translates into action.

0:11:04.6 TG: So things like decision theory associated with that, reinforcement learning, and some of these kinds of more practical applications of those abstract principles. And then there's another level, which is, how does this translate into something which can be actually realized in a physical system in our world, right? Where the principles that come from studying things like artificial neural networks, and correspondingly human brains, are just as important to understanding intelligence, but understanding it in a different way, right? It's less the why do we do the things that we do, and more the how do we do those things, and what's the way that can be instantiated inside a physical system? And then there are lots of open questions that we have that kind of lie in the territory between those two things, right? So, how is it that you can make systems that are able to do something like what we think abstractly thought should be, like using things like neural networks and so on? Where is it that they fall short? How is it that they differ from the kinds of solutions that human minds find? So there's plenty for us to do as cognitive scientists, but I think those most abstract questions are ones that we have some resolution on.

0:12:19.5 SC: But do you have, I know this is probably an extremely unfair question, is there, like, a list of the laws, like in thermodynamics? We have the first law, the second law, things like that, or is that a little bit not quite codified like that?

0:12:34.2 TG: It's not quite codified like that. Yeah, I mean, I think there's still... You know, there are things that we can point to, like, Bayes' rule as a general principle of probability theory that allows us to describe how it is that we should go about making inductive inferences. And you could think about that as something that's a candidate kind of law. And then in logic, there's sort of analogous things like modus ponens, a particular form of argument, right? If P, then Q, P, therefore Q, right? That's a sort of description of a kind of inference that it's valid to make in any circumstance where you can substitute in your P's and Q's, right? And so those are things that are sort of appropriately law-like, but it's more that we have these mathematical systems that allow us to characterize what it is thinking should look like in these different circumstances.

0:13:18.6 SC: Okay, so you've mentioned probability theory, Bayes' theorem, things like that, and we will get there. But I want to sort of impress upon people how important that move was by first talking about what there was before that. I mean, Aristotle, Leibniz, even Frege and the more modern versions of logic, they were really about things that were either true or false, not just having probabilities, right?

0:13:42.3 TG: Yeah, that's right. So, really, the place where the book starts is with Aristotle trying to figure out what makes a good argument, right? And so Aristotle did that by thinking about syllogisms, right? These sort of simple arguments where you'd have two premises and a conclusion. You know, where they're about, a sets of things like, all A's are B's, all B's are C's, therefore all A's are C's, right? That's a sort of classic kind of syllogism. And so he did some theorizing about, first of all, trying to identify what are good syllogisms, and then, second, trying to say, what's the theory of what makes a good syllogism, right? What are properties that we can use to look at these good syllogisms and say what it is that they have in common? And that's why I was saying, he was maybe the first person to really try and develop a little bit more of a theory of what good argument or maybe even good thinking might look like. And the reason why that was important is that, for both Leibniz and Boole, who were the next people who tried to actually formalize thought, what they were trying to do was formalize Aristotle.

0:14:50.1 TG: So their way of saying, "Oh, I've succeeded in coming up with a mathematical theory of thought," the proof of that was going to be that they could reproduce the conclusions that Aristotle had produced about what made something a good syllogism or not. And so they set out to do that, each in slightly different ways. Leibniz had this idea that arithmetic was going to be enough. You know, that was sort of what he had as math, right? And he sort of knew deeply how to do... You know, you could automate arithmetic. He'd sort of built mechanical calculators. And so, if you could express thought in terms of arithmetic, then it was going to be something that you could get a machine to do for you. And he sort of had this vision of how all of this would work, but it didn't quite work out in the system that he had. And then, 100 years later, George Boole came along, and he had a kind of mathematical training that was unusual for an Englishman, sort of having learned all of this continental algebra. And then he recognized that it wasn't quite arithmetic that you needed, it was a slightly different algebra.

0:15:51.5 TG: And then that became something where he was then able to show, using that math that he'd introduced, you could actually take all of Aristotle and sort of start doing it mathematically. And that title, 'The Laws of Thought,' that comes from that 19th-century effort where Boole and his contemporaries were interested in this idea that, yeah, just like you have the laws of nature, you might have this parallel laws of thought. So that's the first thread that I start to trace through to the modern day.

0:16:20.8 SC: I'm kind of curious about Leibniz, because I actually don't know about the aspect of his work that you're referring to, the idea of sort of formalizing thought as arithmetic. So, on the one hand, I can see that it's maybe a precursor to modern computational theories of the mind. On the other hand, I have no idea what he's talking about. Like, isn't Laws of Thought have nothing to do with arithmetic? Like, what was his aspiration there?

0:16:47.4 TG: Yeah, so this is based on a series of unpublished notes that he wrote. It's a really interesting case, if you know, Leibniz was clearly a genius, and this was a problem that he never solved. But you get to see him working through this because there's this series of notes that he writes that are going to be part of his big book, which is about something he called the universal character, right? And the universal character was this idea that you might be able to write things down in such a way that it was clear what the consequences were that followed from those things. And so this was an idea that his contemporaries shared. In the book, I talk about the Reverend Wilkins, who came up with this script and corresponding way of pronouncing it, in which it was impossible to say something which was false. So you could tell from the word for fish and the word for cat that fish and cats were not the same things. So you could never say "a fish is a cat," just because the symbols that you use or the sounds that you would make, corresponded to, sort of, it told you the location of this object within a taxonomy.

0:17:52.1 TG: And so there was this idea that you could sort of systematize language in such a way that then the truth of things would become self-evident. And for Leibniz, he wanted to take that one step further and do that through math. And so what he did was take your Aristotelian syllogisms, and then, with each term within those syllogisms or each set of things, you wanted to associate some numbers. So, he actually... We could credit him with inventing the vector embedding, because he had this idea that there was a string of numbers that you would associate with each thing. And one version of this is just, there's just two numbers, and then...

0:18:36.8 SC: Sorry, the vector embedding is what is used in large language models too.

0:18:40.6 TG: Yeah, that's right. So it's the idea that you can represent a word as a vector of numbers. Yeah. And so he had this idea then, okay, so maybe if you have these numbers for this set, and then these numbers for this set, if the second set of numbers divides the first set of numbers, then we can say that the second set is contained within the first. And so, that sort of gave him a way of thinking about how you could use arithmetic and sort of doing things with prime numbers, and trying to figure out how to make this work. And then he does this, and he sort of like assigns values to various terms and then runs an argument. He's like, "Okay, it works." Then he goes on to the next one in Aristotle, and he's like, "Oh, no, it doesn't work." [laughter] The notes stop, right?

0:19:22.3 SC: Oh, yeah, yeah.

0:19:22.9 TG: And then the next note, a little bit later he gets another argument to work, and so on. But he never quite gets the whole thing to work out.

0:19:29.7 SC: I am completely unfamiliar with that. That's amazing. But I'm very familiar with the spirit of it. Like, clearly, Leibniz was a "theory of everything" guy, right? Like, the one simple trick that will explain the whole universe. This has been something that thinkers throughout history have fallen for.

0:19:46.7 TG: Yeah. But he was also, I think, visionary in recognizing that if he could get this to work out as arithmetic, then thought would be something that we could get machines to do for us, right? And then you can think about everything that's unfolded from there as really trying to find better kinds of math to make this work, with the consequence that, yeah, we can get machines to do some of it on our behalf.

0:20:09.0 SC: And then George Boole is another fascinating character because, again, I don't know enough about the history there, but I think it's maybe easy to underestimate his contribution, because in some sense he just says, "What if all numbers were zero or one, right? Like, what if everything was just true or false and that's all we cared about?"

0:20:26.9 TG: Yeah, it's a little more complicated than that.

0:20:29.5 SC: Oh, sure it is.

0:20:30.5 TG: So if you work your way through. So he wrote two books. The first one he wrote, it's quite short, and he wrote it in sort of like a visionary fit, right? So he actually had this first vision as a teenager wandering through a field in England.

0:20:54.1 SC: Wow.

0:20:54.4 TG: And he sort of has this moment which he really attributed to a divine insight, which was the idea that maybe something like algebra could be used to describe thought. And then he was incredibly busy. He started a school of his own, and he was, for most of his life, a teacher and headmaster, and running the school at the same time as writing these mathematical papers, that were then the highest level of mathematics and receiving a medal from the Royal Society despite never having had a university affiliation. And so that mathematical spirit was expressed in his first book about this, which did exactly what I was saying about trying to turn Aristotle into math. And he sort of had a scheme for doing this, and then it got developed further into this long treatise and investigation of the laws of thought, which the first half is about probability theory, and then the second half is about... Sorry, the first half is about logic, and the second half is about probability theory. And in that book, the place he starts, because of Aristotle, is thinking about sets, and how it is that you could think about expressing the relations between sets in terms of mathematical operations. Right? So if X is a set and Y is a set, then X times Y is going to be what we'd now call the intersection of those sets, the things that are both X and Y. And he sort of works out the math for doing that and then he works out how to extend that math to arguments, right? So things that look a little more like what we think of as logical statements, in terms of P is true, and Q is true, and so on.

0:22:30.2 SC: And did at any point along this development, I know you already mentioned that Boole talked about probability theory, but what was the thought about people thinking, like, okay, this is logic, but actual people in the actual world aren't very logical all the time?

0:22:46.6 TG: Even Boole thought about that. So, I think one thing that's really interesting about Boole was that he kind of thought of himself as a psychologist. There's at least one place where he describes himself as a psychologist. I think in his nomination to the Royal Society, that's how he self-describes. But he was definitely not an empirical psychologist. He was a very theoretical kind of psychologist, if he was a psychologist at all. And so his approach, which he's unapologetic about, he writes saying, there's no need for us to go off and do experiments and figure out what are the laws of thought, because when we write them down, it's self-evident to us that this is a good way of thinking. [chuckle]

0:23:32.3 SC: Okay.

0:23:33.8 TG: We can recognize what good thinking looks like, and then we can sort of capture that with mathematics. And so, he had distanced himself from whatever actually it is that humans do. And that was something that would come back and bite cognitive scientists in the 20th century, right? As the first kind of growth of cognitive science came out of recognizing that it was possible to use something like logic and something like what computers were doing as a way of generating theories about what could be going on inside people's heads. And they ran with that for a little while, but then started to realize, oh, there were lots of things that it didn't describe very well. And that sort of opened the door to then thinking about other theoretical approaches.

0:24:18.9 SC: Well, it's probably a huge oversimplification, but the idea of just taking seriously that we're not sure about things, that we do assign a certain probability to something being a true belief or a false belief, that seems like the next big thing in my head.

0:24:35.5 TG: Yeah. And that was the second half of Boole's book, right? It was talking about uncertain inference. So Boole was very much interested in induction as much as deduction, Right? So deduction is reasoning from certain things to uncertain things... Sorry. Deduction is reasoning from certain things to other certain things. And induction is reasoning from the things we know, which might not be enough to determine what the conclusions are, but still nonetheless sort of making some reasonable inference on that basis, right?

0:25:05.2 SC: Yeah.

0:25:05.8 TG: And so he was interested in induction from the perspective of how it is that scientists figure out the principles of the world around them, where it was clear that they're not doing something like deduction, right? They're not being able to identify a bunch of things that are true and driving their consequences. Maybe they do a little bit of that when they're theorizing, but coming up with the theories themselves and coming up with the sort of generalizations about the world, and even coming up with the laws of nature, right?

0:25:35.0 SC: Yeah.

0:25:35.7 TG: Is something that's an inductive enterprise. And he really wanted to understand that.

0:25:39.7 SC: But it's an inductive enterprise. Let's get this clear, because it confused me for literally decades. I mean, there's a logical kind of induction, which always does get you to certain answers, right? Mathematical induction. But then there's this more informal kind of induction, where you see, well, I see a lot of things happening. Maybe that happens all the time.

0:25:58.7 TG: Yeah, that's right. And again, people in the 19th century were trying to draw those lines and figure that out, right? So you had people like Charles Sanders Peirce, who was trying to work out, you know, here's the way that deduction seems to work. Let's see if we can write sort of similar kinds of schemas for different kinds of inductive arguments. He distinguished between induction, which is kind of seeing instances of things and then going to the general law, and abduction, which is seeing something happen and then coming up with an explanation for it. I would call both of those inductive inferences, but they're both have this sort of like fundamental, there's something uncertain about the conclusion that you're reaching. And then, I think it took a little longer to really start to be able to use the solutions to that are offered by probability theory as a tool for understanding how it is that people make inductive inferences. And again, that's a kind of 20th century innovation.

0:26:55.6 SC: I'm a huge fan of abduction. I'm not sure if I'm more of a fan than I should be, but it seems to me to be the closest to the way that science actually works, right? And and sometimes it's mixed up with inference to the best explanation, and it's sort of admitting that this is not clear-cut and algorithmic, but still there is something uniquely sensible that we're doing.

0:27:18.5 TG: Yeah, I think that's right. I mean, I think, for many of these kinds of inductive inferences, probability theory gives us a good description of how it is that you should make those inferences. But for something like abduction, a lot of the work is actually done at the level of how people really do make those inferences, right? Like, one of the challenges is you're probably coming up with a kind of thing that no one's thought about before, right? You have to come up with the hypothesis in order to be able to entertain that hypothesis. And that's something which is a algorithmic level phenomenon, right?

0:27:58.3 SC: Right.

0:27:58.8 TG: It's something which is about something our brains are doing rather than something we're told how to do using the math.

0:28:04.2 SC: And this is skipping way ahead to the future of our conversation, but let me just bring it in right now. Like, I have this feeling that a large language model, or some machine learning algorithm that we have right now, would be perfectly good at solving Einstein's equations of general relativity in the right context, but it would really struggle to have that first creative moment where it it suggested that gravity is the curvature of spacetime, because simply because they are trained on things that have already happened. Is this me being anthropocentric, or do you think that there's some truth there?

0:28:38.6 TG: I think there's some truth to that. It's certainly something which I think is an open question about the capacities of these models, in terms of the extent to which they're able to make those sorts of extrapolative inferences. And we would expect that they could do so to the extent that again, what they've been trained to do provides some kind of infrastructure for being able to do that, right? So we find that there are certain kinds of creative thinking that these models can actually do reasonably well, like, they're maybe better than people at coming up with simple kinds of analogies, right? And you can think about that as being a consequence of having this very fine understanding of language, right? That's important for doing that. But analogy is an important part of discovery, too. Sometimes you make a discovery by recognizing that an idea from one domain applies in another domain. And so I think it's not going to be as simple as they're not able to do this thing that we think of as abductive inference. I think it's going to be something where, there's going to be a sort of set of things that they're going to be able to do well because they align with the kinds of tasks they're trying to do, and then a set of things that maybe are harder for them, because they push against that training.

0:29:46.8 SC: Yeah, okay, that does make sense. And back to the probabilities and the beliefs. My podcast listeners hear me talk about Bayesian reasoning all the time. Is that what we're talking about here? And I know that Bayes himself just sort of, I guess, was it even posthumously published, his formula? So he was not a big player in that discussion, but we give him some credit.

0:30:08.3 TG: Yeah, so we talked about one thread, which is the thread of logic through Leibniz and Boole. And then the book is really about three threads of thinking. So one is logic, one is sort of foundations of neural networks, which I sort of characterize in terms of spaces, features and networks. So thinking about thoughts as corresponding to a point in space, right? And some of the mathematics that we use for thinking about spaces, like calculus and so on, being a tool for then thinking about how thoughts work. And then the third thread is this thread of probability theory, which I think each of them is very helpful and sort of complementary to the other in explaining various aspects of how thinking works. And so for probability theory, yeah, the origin that I focus on in the book is this sort of 18th century idea, where really the radical idea is that probability theory could be applied to thought, right? So before that, people had been developing probability theory as a kind of mathematical theory. And it was a mathematical theory that applied to things like gambling games, right? So you see this in the very earliest origins of probability theory.

0:31:18.5 TG: The best example we have is Gerolamo Cardano, who is a mathematician, but also an addictive gambler. And so he really wants to know, for his recreational and financial reasons, how to think about the outcomes of rolling dice or these sort of probabilistic events. And so he works out the mathematics of how to do that. And there's a few... The next sort of moment that we see in the origins of probability theory is Blaise Pascal doing something similar, right? Sort of like trying to solve a gambling problem and from that developing some of the foundations of probability theory. But that was a theory of what happens when you roll dice, right?

0:32:07.0 SC: Right.

0:32:07.6 TG: And the innovation that comes with Bayes was saying, well, maybe this mathematical system that we have, this sort of set of axioms that characterizes some mathematical object, also characterizes another thing that we're interested in. It's not just what's going on with dice, it's also what's going on inside our heads when we change our beliefs, right? So he was thinking about some gambling inspired examples, right? So, if there is a lottery which is paying off at some rate, how do you estimate the rate at which it's paying off? But the way that he sets that up is in terms of the beliefs that you have about that lottery. Like what's a reasonable estimate that you should have for the probability that it's going to pay off at the next moment, given the examples that you've seen so far? And then that idea is developed further by Pierre-Simon Laplace, who really sort of came up with it independently, and really, worked out all of the consequences of that way of thinking about how to update our beliefs.

0:33:09.0 SC: So would it be fair to say that, at least roughly speaking, pre-Bayes and Laplace, you could gamble, there could be uncertainties and things like that. But when it came to thinking, you were supposed to be right or wrong. And after them, you could say, well, I have a certain degree of confidence in my beliefs.

0:33:28.7 TG: I think there's a discrete point that happens with Bayes and Laplace just in sort of working out that theory. But you do see hints of people talking in those terms before that, right? So even Wilkins, who came up with this sort of idea of the language that you could use where you could never say anything false, talked about probability as a way of talking about a degree of belief. He didn't work out the mathematical consequences, but he used that language. And Pascal, famously, after he departed the world of mathematics and instead started to think about religion, made a probabilistic argument in that setting, which is really an argument about belief as well. So you see lots of hints of this prior to Bayes and Laplace, but I think they're the ones who really developed that into a theory of what we could call thought.

0:34:18.4 SC: So let's dig into that theory of what we could call thought. We have some beliefs, they're probabilistic. How do we reason with them in the way that Boole would have had us reasoning with true and false statements?

0:34:31.8 TG: I think the really cool thing about Bayesian probability is that one very natural way to see it is just an extension of logic, right? So in logic, we talk about possible worlds, right? So if you have two propositions, P and Q, you can imagine all of the possible worlds that you could be in. You could be in a world where P is true and Q is true, a world where P is true and Q is false, a world where, P is false and Q is true, and a world where both P and Q are false. Right? Those are the possible worlds we could live in. And logic is really about what conclusions you can draw with certainty based on the information you have about what world you might be in. So if you have got enough information to rule out some of those possible worlds, such that it has to be the case that the world you're in is one where Q is true, then it's reasonable to conclude that Q is true. And so your classic logical arguments are arguments that are telling you, "Oh, the information you have gives you enough constraints on the worlds that you might be in that this is a reasonable conclusion that you can draw about that world."

0:35:31.5 TG: Probability theory takes one more step, which is to say, for those possible worlds, we're also going to assign a number to them. And that number reflects our degree of belief about the probability that that world is true. And then as soon as you do that and you start following the rules of probability theory, you're then updating your beliefs about, you know, what world is it that we're likely to be in based on the information that I've got so far. And so the logical arguments still work. Right? So the... You know, you can think about a logical argument is telling you that something is true with probability one, that with certainty, it has to be the case that you're in a world where this thing is true. But it's generalized by probability theory in allowing us to say, "Oh, we don't have enough information to determine that this thing is true with certainty." But we can say, "Oh, well, there's like a 70% chance this thing is true." But the same kind of idea of as you get information, you're maybe ruling out some possible worlds, maybe... And as a consequence of that, you're changing the probabilities of the other worlds that you could be in, or you're getting information that changes the chances that you think you're in one world or another. And then you're just doing the same kind of thing. It's just that you're now doing it in this much more graded way.

0:36:46.1 SC: And just like we can question how conventionally logical people are, we can also question how good they are at updating their beliefs when new data come in. I mean, so maybe should we think about perfect Bayesian reasoning as once again aspirational when it comes to the laws of thought? Or is this meant to be a description of how people actually think?

0:37:10.2 TG: No, it's a tool at that abstract computational level of saying, what is it that we should be doing? Right?

0:37:17.5 SC: Yeah.

0:37:18.2 TG: What's the solution, the ideal solution to the problem that our minds face? And so when our minds face inductive problems, probability theory tells us what the ideal solution to those problems look like. And of course, there's lots of ways that that doesn't line up with the things that people actually do. And in the 20th century, we started to explore those. Some of those, I think, are things that we can explain from this sort of Bayesian perspective, but where we have to think about people doing something slightly different from the thing that they've been told to do. I can talk a little more about that. And then some of them are in between these different levels of analysis. Right? So we have that abstract computational level, what should you be doing? And then there's the algorithmic level, which is what's a good strategy for trying to get close to that. And you can ask a question at that level which is about, what's the best that you could do at trying to do the thing you're supposed to be doing with particular constraints on the resources that are available to you.

0:38:17.1 SC: Right.

0:38:17.2 TG: And then when we characterize what those constraints are, we can actually work out what that looks like. And in many cases, that actually gives us insight into some of the, you know, what might seem like strange things that people do. And that's something that I've done a bunch of work on. And we actually have a book coming out in a few weeks, which is about that idea of, we call it resource rationality. And the book explores how to change our notions of rationality as a consequence of recognizing those kinds of constraints.

0:38:45.2 SC: So you have two books coming out like within a month of each other?

0:38:48.2 TG: It's very awkward. They're coming out within a week of one another. One is more for a general audience and sort of exploring these kinds of ideas about the laws of thought. And then the other is a slightly more academic book, which is focused on this idea of what we should do with our limited cognitive resources.

0:39:09.1 SC: You can still do book signings where you sign both at once. That's okay. That makes perfect sense. So, in other words, can we sort of simplify it down to we would like to use Bayes' theorem to update our beliefs, but that's hard and that takes a lot of resources. So evolution and biology have equipped us with certain shortcuts?

0:39:31.4 TG: I think the way that I would think about it is, yeah, I think it's that... One of the interesting paradoxes of human cognition is that we are both, from the perspective of a psychologist, error-prone decision-makers who sort of use these heuristics that result in biases. And from the perspective of computer scientists, these aspirational agents that are doing the kinds of things that we'd like our AI systems to do. And so if you want to resolve that paradox, the way that I resolve it is to say we are good at solving the kinds of problems that we face with the resources that we have. Right. And you can think about that as being the consequences of different kinds of adaptation. So evolution as well as learning. So over the course of our lifetime, learning to use our cognitive resources better, right? As well as just sort of engaging in some planning, or we call it meta-reasoning, about how to appropriately approach different kinds of problems that we're trying to solve.

0:40:35.3 SC: We did have Karl Friston on the podcast some time ago talking about the free energy principle. Is that an example of the brain trying to solve these hard problems in an efficient way?

0:40:48.0 TG: That's a good question. I haven't thought about it in those terms. I think the way that he sets that up is more in terms of a kind of objective that the system has rather than in terms of a resource constraint. And the way that we think about it is a little more explicitly saying if you are... If we want to redefine what rationality is in a way that works for agents with finite computational resources. This is drawing on an idea from the AI literature from Stuart Russell and Eric Horvitz. The way to define what rationality is for a bounded agent is more in terms of taking... Instead of focusing on sort of taking the action that's the action that probability theory and so on tells you should take, it's using the best algorithm to choose the action that you're going to take. And so it's sort of popping up a level of abstraction in terms of thinking about defining rationality at that meta-level as a tool for then generating what are the appropriate ways of using your cognitive resources at the object level, the actions you take in the world. And my previous book, 'Algorithms to Live By,' is actually a pretty good general audience treatment of those ideas. We didn't express it in terms of this framework of resource rationality, but it's really about the idea that in some ways computer science provides a better guide to rationality than economics, right? Sort of thinking in terms of probabilities and rewards.

0:42:20.8 SC: I mean, I guess I've never really thought about it this way, but when you are computer programming, the resource limitations are obvious. You have a certain amount of time, certain amount of memory, right? A certain amount of whatever data. But of course the same things are going to apply to human brains where we can imagine ideal thoughts, but actually having them is something that's going to be probably too resource intensive.

0:42:45.1 TG: Yep. Yeah, I think that's a reason why I think about it.

0:42:48.0 SC: And so what is your... I mean, you've said a little bit about this, but what do we do? What are our strategies? Living in an imperfect world, not being able to be exact Bayesians at all times. Do we have... What are the shortcuts that actual human people use?

0:43:05.5 TG: Yeah, so some of the kinds of strategies we use are the kinds of things people have identified as heuristics, right? Heuristic just means a rule of thumb or a shortcut for solving a problem. I think part of what is valuable about reanalyzing those from the perspective of resource rationality is being able to say that using those heuristics isn't necessarily a bad thing, right? And the biases that come from those might not necessarily be things that you can avoid given the cognitive resources that you're operating with. So instead we can ask a question like, are we sort of doing the best job we could with the cognitive resources that we have? And then is there a way that we could mitigate those biases by maybe using a different heuristic in a particular setting or something like that? And so the kinds of strategies that we focus on in our work-on resource rationality are things like sampling strategies for approximating Bayesian inference. So instead of thinking about a whole probability distribution, you might think about a few samples, a few possible instances, right? Instead of when you're making a decision, considering all of the possible outcomes, you might think about a few possible outcomes, right? And things like that.

0:44:15.9 TG: And we can look at the kinds of strategies that people seem to use when they do that, when they don't consider all the possibilities, are the ones that they're considering, the ones that they should be considering from the perspective of using limited resources. And then the other kinds of things that we think about are things like setting goals or subgoals. So I think from the perspective of the sort of history of thinking about cognition, folks like Alan Newell and Herb Simon sort of introduced this idea of, we could think about when we're solving challenging problems, we need to decompose those problems into parts. And part of what it is to be smart is to be able to decompose them in those ways. But in many ways, that ability to kind of break down problems and set goals is really a consequence of a resource constraint. So, if you had infinite cognitive resources, you would never need to set goals, because you can just reason all the way to the end of the trajectory of whatever is going to arise from the choices that you're going to make. And so setting goals and subgoals and so on is a tool for being able to make progress on problems with finite cognitive resources. And then we can ask, what are the good goals to set? What's a good structure to sort of give to a problem from that perspective of resource rationality?

0:45:30.1 SC: So... Sorry, in some sense, we all have a goal that could be idealized as live the best possible life. But you're saying that as a strategy for getting there, that's not really reasonable. We can't actually be Dr. Strange in the Avengers and go through every possible part of the multiverse. We have to sort of have subgoals along the way that give us an approximately pretty good trajectory.

0:45:51.9 TG: Yeah. And then you can ask... You obviously don't want to make your subgoal too close, and you don't want to make it too far away. And so the question of where people should set their subgoals and do they do a good job of setting those subgoals is a question that we can engage with from that perspective of resource rationality.

0:46:08.2 SC: There's always this question, Bayesian reasoning, because the whole picture is you have some prior probabilities, you get some data, you calculate a likelihood function, you update your priors. But so then where did the priors come from? Is that question involved here? Like, where do human beings actually have their rough feelings about the plausibility of different propositions?

0:46:29.4 TG: Yeah. So this is, I think, a very deep question, and I think there's different ways that you can ask it. So there's one way of thinking about. So when you think about Bayes rule as our tool for describing what ideal solutions to inductive problems look like, that characterization applies both to, an inference that you might make in perception when you're trying to interpret the light that's falling on your retina. Your brain has to do something that looks like an inductive inference, right? To figure out the structure of the world out there. To interpreting a sentence that somebody says, where you're taking the words that you hear or the sound that's hitting your eardrum and sort of turning that into an inference about what it is that the person said and maybe what they meant. But also to like fundamental things like how do we learn language in the first place? How is it that brains come to be able to interpret the structure of the physical world around us? Right? So all of those things are things you can think about as inductive problems. And so asking where the priors come from is going to be different in those different cases.

0:47:40.2 TG: So the more fundamental case, the one which is about like, how do we learn language, if we think about that as a problem of inductive inference, the priors there are going to reflect whatever the innate predispositions we have to learn language, but also all of the other sources of information that we have that are not the linguistic input. So experience that we have in the world and so on is stuff that's going to inform the way that we learn language from the utterances that we hear. And so that is a good tool for using, for thinking about what are differences between like humans and large language models, where the big difference between human minds and brains and large language models that we have today is about inductive bias. It's about being able to learn from the small amounts of data that we get as humans relative to the very large amounts of data that our large language models are trained on. So a human child learns to use language in about five years of exposure, by comparison, that he used to train large language models is the equivalent of between 5,000 and 50,000 years of continuous speech, right? So it's just sort of orders of magnitude difference.

0:48:50.0 TG: And the thing that makes up that gap is inductive bias. It's the sort of the thing that comes from our prior distributions broadly construed, as human beings, that allows us to close that gap. When we look at the everyday inferences that we make, these sort of short-term things like interpreting a sentence or making sense of visual information, those priors are things that are really a consequence of those learning processes having worked, right? It's that we've built models of the world around us that inform the way that we interpret the data that we experience. And that's something where I think it's a little less mysterious where those priors come from, because they come from the world. But they also come from the world plus, again, whatever our sort of like more general inductive biases as learners.

0:49:43.0 SC: So we're not blank slates, right? I mean, I guess various thinkers from Kant to Noam Chomsky have said that like, yeah, we're born with some ideas in our heads and presumably we're a lot better now in the 21st century at teasing out which of the ideas we do come born with and which we pick up along the way.

0:50:05.4 TG: Yeah, I think this is getting into sort of 20th century cognitive science. So we made a leap from the 19th century, which is our logic and probability theory, to sort of like 21st century considerations about, are people Bayesian in the right ways and resource rationality and so on. The missing chunk there is the 20th century, which is where people began to use these mathematical ideas as a tool for trying to understand human minds. And so the first half of the 20th century, psychology was really focused on trying to be a sort of rigorous scientific discipline, having gone from its foundations, which were really introspective, where you would be asking people, whether they saw something or heard something or what the impression of it was. There was a sort of reaction against that. And behaviorist psychologists said, "No, no, no, we can't see a thought or touch a feeling. Let's focus on the things we can see or touch, which are environments and the behaviors that they produce." And so not allowed to really sort of talk about those mental states as explanatory things in accounting for human behavior. And then the advent of computers and the existence of mathematical frameworks for thinking about how to do things on computers and these sort of extensions to logic and so on, which came out of that.

0:51:36.7 TG: That provided a new set of theoretical tools that psychologists could use to come up with rigorous theories of how minds work. So you can talk about thoughts, maybe we haven't quite got to feelings yet, but thoughts, if you have a precise mathematical device like logic for then coming up with hypotheses about what it is that thought does. And so the big sort of successes of that enterprise were Alan Newell and Herbert Simon creating the logic theorist, which was a machine that could discover proofs for mathematical propositions in logic, and Noam Chomsky showing that thinking about formal languages gave us a tool for then making sense of sort of the natural languages that humans use. And those ideas really provided the foundation of science. But they also led to some interesting challenges. So Chomsky's approach to language was very good in sort of illustrating that language was a much more complex object than behaviorists had assumed, that you needed to have kind of like internal structures and things like verb phrases and noun phrases and things that sort of look like grammars in order to account for the structure of human languages. But then it created this new problem, which was, how is it that human beings could possibly learn these very complex objects from the limited data that they get?

0:53:05.9 TG: And so that's the thing that then pushed Chomsky to say, well, maybe they're not really learning it, maybe they're acquiring it as a consequence of having some very strong constraints on what it is that they can learn as languages that then only require relatively small amounts of data to determine, oh, okay, it's this particular configuration of bits and pieces that characterizes the language that I'm actually speaking here.

0:53:30.6 SC: And is there a thought that we should design our AIs similarly? I mean, it seems like the lesson from connectionist approaches to AIs that have led to large language models, et cetera, is the human beings have done all that work. We can just let the AIs be blank slates and train them on a huge amount of data.

0:53:51.4 TG: Yeah, so that's exactly the contrast, is that the AI models give us a really good illustration of how much language you need to solve the problem that Chomsky had identified to learn something that is this very complex object. So if we sort of take as given that they've learned something like human language, then you can think about that as a proof of the point that Chomsky was making, that you're going to need lots and lots and lots of data to learn something like human language. Right? So he was right that in the five years that the kid gets, they're not going to be able to figure out the structure of language. And in fact, it turns out you need something more like 5,000 or 50,000 years of data in order to do a really good job. And so I think that's a really nice way of thinking about what that difference is. And it also gives us a good way of thinking about what the challenge is then. If you wanted to make systems that are more human-like in their ability to learn. So you can think about it. If you want to be able to learn from five years of data, and you're currently learning from 5,000 years of data, then you've got 4,995 years to make up in terms of the content of that inductive bias or those prior distributions that are inside the child's head.

0:55:02.5 TG: And so you can think about that coming from these other kinds of sources. So evolution is one of those as well as some other things. So the broader set of experiences that the child has as they're learning language. Sort of like building a model of the world around them, that means that the things that they're learning aren't just sort of arbitrary sequences of words, but actually things that map onto things that are meaningful. So your large language model has to figure out everything that it knows about the world just from the sequences of words that it's seeing. And also the kinds of things that a child is getting, not just as a consequence of whatever those evolved constraints are, and not just as a consequence of their broader experience, but also the things that they're getting from being able to engage in using that language to produce desirable outcomes in the world around them, right? Using it as a tool, not just something that you're necessarily learning to predict. We do a little bit of that in our training of large language models at the end.

0:55:41.8 TG: There's some sort of fine-tuning about reinforcement learning and so on. But I think that's a really interesting kind of project for cognitive scientists is thinking about, how to characterize that that gap that we have, right? And using these sorts of models as a tool for working that out. And so in in my lab, we've done a bit of work, this is most recently with Tom McCoy, who's now at Yale, looking at an approach that's called meta-learning for training neural networks. And what meta-learning does is it tries to create neural networks where we manipulate the initial weights that the neural network has in such a way that it's able to learn from less data. And so the way that this works...

0:57:01.1 SC: Giving it a head start in a little sense.

0:57:02.9 TG: Yeah, that's right. And also coming closer to capturing those sort of inductive biases and prior distributions, right? So, the way that it works, you say, I've got a bunch of different learning problems I want to solve. In the linguistic case, you could think about this as, "I'm going to want to learn lots of different languages. I'm going to want to learn English, I'm going to want to learn Korean, I want to learn Urdu, I want to learn..." Each of the languages that you want to be able to learn. And you know that you're only going to be able to learn those from your five years of input, right? And you say, what are initial weights that I can put in my neural network that are going to help that neural network learn each of these languages from that five years of input just using the sort of mechanisms that it has for learning. And so the way that we solve that problem, using an algorithm that's called model agnostic meta-learning, is you have a learning process which has an outer loop and an inner loop. And the inner loop is just learning an individual language. So you're just adjusting the weights of the neural network away from those initial weights to learn the each of those languages.

0:58:08.1 TG: But the outer loop is saying, when I look at my performance across all of those languages that I want to learn, how does my performance on those languages change when I change my initial weights, right? And so you can actually learn the initial weights by trying to find initial weights that help to improve performance across all of the languages. And so by doing that, you're finding a starting point for your neural networks, which is one that's going to allow them to learn quickly, right? From the limited data that they're getting. And then we can go back and we can look at that set of initial weights and we can say, oh, what does that tell us about the biases that human learners might have, right? What are the kind of biases that you need to have in order to be able to learn language from the amount of data that we actually get?

0:58:55.0 SC: So my limited knowledge of this stuff goes back to AlphaGo and AlphaZero, that was the chess playing program. And I'm told that those programs did better if they never were exposed to human chess players and Go players and just learned it themselves. So is there a worry analogously that your version where you can sort of do a little bit of a shortcut to give the models a head start by initializing them in a certain way, will that make them less creative in some way?

0:59:31.3 TG: I think it will make them less inclined to find solutions that are not like the human solutions, right?

0:59:39.7 SC: Right.

0:59:40.7 TG: And that's a plus and a minus.

0:59:43.5 SC: Pros and cons there. Yeah, exactly.

0:59:45.0 TG: Yeah, that's right. Because there are lots of things that humans are really bad at, right?

0:59:48.6 SC: Right.

0:59:49.1 TG: And we might want to be able to make AI systems that can complement us by being able to do the things that people are bad at. And that's a really, I think, good way of thinking about what a possible future is where humans and AI get to exist side by side in a way which is good for everybody. I think the thing that might be quite good about making AI systems that have inductive biases that are more aligned with people is that it's going to not only make it possible for those neural networks to learn from less data, and so some of the energy concerns and so on that are involved in training those models might, you know, get better and maybe they'll actually be able to learn even more impressive things when you give them the 5,000 years of data, right? But it also means that those systems are going to make more sense to humans, right?

1:00:35.5 SC: Right.

1:00:36.1 TG: So one of the kind of... The two things I would say are the big differences that we see between human minds and our current AI systems are one of these is about an inductive bias, right? Ability to learn from small amounts of data. And the other is about generalizability, where you can have an AI system that's very good at solving one problem and then fails quite spectacularly on a problem that's right next to it, right? I think we've all had that experience of like, the AI system seems very smart and then does something very weird, right? And this has been called jagged intelligence by various people and that's a sort of interesting frontier for AI researchers to try and figure out is how do we understand what those jagged boundaries look like and how do we make our systems better? I think cognitive science is actually a really good tool for trying to answer those kinds of questions. But one thing that might happen is that if we create AI systems that have inductive biases that are more similar to people, then the solutions that they're going to find when they're trained on the data that they get will be more like the kinds of solutions that humans find too. So you can kind of think about it as your AI system is your blank slate. Okay, it takes 5,000 years of speech to get it to the point where your five-year-old gets to.

1:01:26.0 TG: But that might not actually be the same point, right? It might from the outside look kind of similar and that they're both doing a good job of sort of using and producing language. But on the inside it might be quite different and there's some weird path that it's found that gets it to the point where it's able to do a good job of using language, but it's kind of just a very weird solution from our perspective. And in fact, some of the analyses we've done in my lab suggest that that's the case. And so if we can use inductive bias to nudge the models towards more human-like solutions, they're probably going to be things that make a little more sense to us as well.

1:02:23.7 SC: Well, I want to hear more about that little parenthesis you just said. I mean, I always presumed that the internal machinations of the LLMs that output a human-sounding sentence were very, very different than what goes on in an actual human brain. So what do we know about that?

1:02:41.3 TG: Yeah, so some examples of things that we've done in my lab that sort of reveal some of this weirdness. One of them is that large language models are very sensitive to the probabilities of the outputs that they're producing, right? So when people were very excited about these models, there was the paper, The Sparks of AGI paper that came out that said GPT4 sort of exhibits these remarkable abilities. Tom McCoy and some colleagues, we wrote a paper that we called Embers of Autoregression, which was saying, as much as you're getting sparks at the top, there are still these embers at the bottom, which are a consequence of the way these models are trained. And so one of these is that if you... And again, these are things that in modern systems there's all sorts of tricks that they've used to sort of get around this. But if you sort of take a raw language model of the kind that we were getting with GPT4, and you ask it to solve sort of simple problems like counting the number of letters that appears in a string, how well they do on that is influenced by the probability of the answer that they would have to produce.

1:03:55.2 TG: So, for example, they're much better at counting strings that have 30 letters in them than strings that have 29, because the number 30 appears on the internet more often than the number 29. So it's a situation where there are other nearby answers that are pretty good and some of those have higher probability. And so as a consequence, it sort of produces the high probability thing rather than the thing that it's supposed to produce. And so that's a sort of like weird idiosyncratic bias of language models. It's a consequence of the way that they're trained. And so more generally, the way that I think about these systems is that we should expect, this is applying our computational level lens, right? We should expect intelligent systems to behave in ways that are shaped by the kinds of problems that they're trying to solve. And when we design our AI systems, we're making explicit choices about the kinds of problems that they're going to solve. Things like being able to predict the next word or token that appears in a sequence, and that's going to be something which influences its behavior. And so to the extent that there's a difference in the objective function, right, the goal that we have in training that system, and the kinds of computational problems that human minds have evolved to solve, then we're going to expect the kinds of solutions that they find to look quite different. And that's part of where we get this mismatch in behavior.

1:05:12.2 SC: You mentioned earlier this provocative thing which I haven't had a chance to follow up on about, was it the geometry or the spatial structure of neural networks or neurons more generally and the role that that plays in the laws of thought?

1:05:28.4 TG: So, the third thread, so we've talked about logic and probability theory, right?

1:05:32.0 SC: Yeah.

1:05:33.0 TG: So the third thread here is this kind of idea of thinking about thought in terms of points in space and using the sort of math that we use for thinking about spaces. And that was an idea that developed in the 20th century, right? So I said we had Newell and Simon and Chomsky demonstrating that things like logic were really effective for giving us sort of ways of expressing theories about how something like thought or language might work. But then that was sort of like the big idea of the 1950s and carried forward from there. And then in the 1970s, psychologists sort of started to realize that there are some gaps in this. And so this is where we get to, okay, what happens when you take these mathematical theories like logic and then start comparing them rigorously against human behavior? And when you start to do that, you start to turn up these sort of meaningful discrepancies. And so one of these sets of discrepancies came from the work of Eleanor Roche, who was a psychologist who explored how people think about categories.

1:06:36.1 TG: If you think about categories from the perspective of logic, you're looking for a rule that characterizes that category. You're looking for like a definition that tells you exactly what it is to be a member of that category. You have to have these properties. You have to not have these properties, whatever it is. That's the rule that tells you what it is to belong to a category. And Roche showed that it really seems like very few human categories have that kind of structure. So if you look at our intuitions about what makes something a piece of furniture or what makes something a vehicle, there aren't definitions that you can find that characterize those in the way that the logician would want you to have. And instead, it seems like they're characterized by a much more fuzzy structure where you can say, certain things are definitely pieces of furniture, like an armchair, and other things are maybe pieces of furniture like a rug, right? And there's this sort of gradients that was hard to capture from that logical perspective. And so it seemed like you needed a different kind of theory to be able to capture that.

1:07:35.7 TG: And psychologists started to think about, well, maybe if you think about objects as points in space, then how close things are. And space is a way of characterizing the extent... If you think about, you have another point that characterizes your category of furniture. And so now an armchair is close to that point, and a rug is further away from that point. And maybe that gives us a way of capturing that sort of gradients. But then you end up with a new problem, which is, if concepts are points in space, then how do you do something like computation? So with logic, that translated into the ideas behind digital computers, Turing machines, all of these things. We had a way of thinking about what thought was, because we could say, okay, if you represent something logically, then fulfilling that idea of Leibniz, we can then make a machine that executes our rules and sort of tells us what the consequences are. But if concepts are point in space, then where do we go from there?

1:08:36.2 SC: What do we do with that?

1:08:37.3 TG: How do we learn what those concepts are? And how do we sort of work out what the consequences are? And the answer to that came from neural networks. So people had been thinking about neural networks since the 1940s. There was this sort of initial work by McCulloch and Pitts, which was translating the idea of a Boolean circuit. The kinds of logical structures that George Boole had thought about, they came up with a way of expressing Boolean circuits in terms of operations between neurons. You could connect neurons up in such a way that they could represent logical ands and ors and nots and so on. And from that, you could build sort of complex neural circuits. And then Marvin Minsky had actually, as a doctoral student at Princeton, his PhD thesis was all about neural networks. And he actually built a neural network that could learn in the basement of the psychology department at Harvard using sort of bits that he was able to scrounge in places, and then decided to give up on that whole thread of research because he thought it would be impossible to build a neural network that was large enough to learn anything interesting.

1:09:47.5 SC: Right.

1:09:48.1 TG: Right? So with his adjustable resistors and these things that he was using for building his neural networks. There was a very clear constraint on the size of the model he could build. And he sort of did some calculations. He's like, "Oh, in order to learn anything interesting, it would have to be ridiculously large." And so he abandons that. And then Frank Rosenblatt, who was a psychology PhD at Cornell, in his PhD got into building computers. He actually came up with a device for tabulating data electronically as part of his PhD thesis and sort of started to think, "Oh, hey, this thing could maybe help us understand how brains work." Wanted to understand how brains work, and then came up with the first provably effective learning algorithm for simple neural networks, something called the Perceptron, which is a sort of a neural network which has one layer of adjustable weights in the original version that he had, although he subsequently explored versions that had multiple layers of adjustable weights. And so you can think about what a Perceptron does, what a neural network does, as you've got your inputs that go into that neural network. So you've got a set of units that represent different dimensions of the input. You've got an output that comes from that neural network. But what it's doing is it's taking a vector of values that goes in and it's transforming it into another vector of values. Right? So you can think about a neural network as precisely as a way of computing with spaces. It takes a point in one space and then it transforms it to a point in another space.

1:11:22.8 SC: Got it. Yes. Right.

1:11:24.0 TG: And so as you add more layers into that neural network, then the kind of computation that it's able to do gets sort of more complicated. Now you're able to do multi-step computations where you're transforming point from this space into the point in this space, and a point in that space, and a point in another space, and so on. And so you're able to solve sort of potentially more complex kinds of problems. And so that approach had kind of hit a stumbling block because after Rosenblatt had shown, okay, we can have a learning algorithm, his learning algorithm sort of only provably worked for neural networks with one layer of adjustable weights. And so Marvin Minsky, working with Seymour Papert, wrote a book that said, in fact, those neural networks with one layer of adjustable weights are fundamentally limited because they're limited to being able to compute things that are linear functions, that are local functions of their input, then they're not going to be able to solve interesting kinds of problems. And they sort of characterized a bunch of interesting problems that the models couldn't solve. And so people got a little bit less interested in neural networks.

1:12:38.1 TG: And so the computer scientists who sort of got excited about this drifted away. And another bunch of psychologists, David Rumelhart, working with Geoff Hinton, who was a postdoc in that group, Jay McClelland, they started exploring these models and then came up with learning algorithms for multilayer neural networks that could then solve these more complex problems. And then as a consequence, like if you fast forward another 40 years, we get to our modern AI systems. And so the big thing that changed over those 40 years was just being able to use these models at much larger scales, right? So deeper neural networks trained on more data, with many more internal connections and so on. And in many ways, you can kind of think about that as Minsky there in the basement of the psychology department, saying, this is never going to work because you'd have to run it at ridiculous scales. Really, you can see that the current reality is people just accepted that ridiculously large neural networks are what you need and are sort of willing to pay for having those and the sort of technologies there to be able to create them.

1:13:47.8 SC: Certainly from physics, there's a historical lesson that if there is a mathematical theorem saying you can't do something, you will always become famous by figuring out how to do it somehow by violating the assumptions of the theorem one way or the other.

1:14:01.8 TG: Yeah. One thing that I really like about that story, though, is that the key insight that made it possible to train these multilayer neural networks actually comes from Leibniz. So the piece of math that you need to derive the backpropagation algorithm, which was actually named by Rosenblatt, he sort of had a backpropagation algorithm. It just didn't work quite well. So the backpropagation algorithm that Rumelhart and Hinton developed was using the chain rule...

1:14:35.4 SC: There you go.

1:14:35.7 TG: Which was a piece of the calculus that we can trace back to Leibniz.

1:14:38.8 SC: I mean, maybe a good place to wind up the conversation is to return to what you mentioned at the beginning of our conversation, at the end of your book, which is Marr's level of thinking. There's sort of a nice synthetic way of thinking about the reason why there's not just a simple once and for all set of laws of thought. There's a lot of things going on. They're all interconnected with each other.

1:15:03.2 TG: Yeah. So the way that Marr set this up, so we have these three levels, right? The computational level, that's about the abstract problem being solved and the ideal solution, the algorithmic level, which is about the sort of actual processes that approximate that solution, and the implementation level of how that's realized in something physical. That tells us that we're never going to have just one theory of how the mind works, right? Because you can have theories at those different levels which are all correct as long as they're compatible with one another, right? So we can say, "Oh, at the computational level, logic and probability theory really give us a good answer for what it is that minds and brain should be doing." And then at the algorithmic level, you can say, "Oh, an artificial neural network gives us a way of approximating the solutions to those systems. And then at the implementation level, we have a sort of story about, "Oh, well, this is how that kind of structure could be realized in something which is either cells in a brain or pieces of silicon." right? And so at those levels, as long as those theories are compatible, everyone can be right.

1:16:07.4 TG: And I think that's really important because there's a long history of people wanting to have a sort of single explanation for things, right? Where you're like, oh, this is the theory, this is the one thing which is true. And I think when we come to think about information processing systems, it's really clear that we're going to have to have these nice sort of mutually reinforcing perspectives that give us explanations at these different levels of analysis. And what it really means to have a complete answer is understanding those different levels and then how the explanations we have at those different levels relate to one another.

1:16:39.5 SC: Yeah. I think of it as a story of emergence, but there's more than one useful true way of talking about the universe.

1:16:45.9 TG: Yeah. I think the other thing that's important is that you can also... There's not necessarily a one-to-one mapping between those levels, right?

1:16:53.4 SC: Yup.

1:16:53.8 TG: So you can have Bayesian inference be your computational level explanation for something. And you can have five different ways of creating approximations to Bayesian inference that all show up in human minds and brains in different places. And so, then, you know, those things are going to be realized in brains, in sort of neurons in different ways as well, right? You can come up with different ways of instantiating those different algorithms. And so, I think that's also important when we try and look for sort of like silver bullet explanations where you're like, this is the one way that the brain does Bayesian inference or whatever it is, right? My expectation is that it's unlikely that we'll find those, just because there are so many ways of approximating those ideal solutions, and different ones work in different circumstances. That's kind of what keeps statisticians and computer scientists busy coming up with new kinds of algorithms. And so, when we think about something like the laws of thought, the level at which I think we can be most successful is at that computational level. I think that's where we can really say, these are the principles that really govern intelligence, whether it's in humans or machines or in aliens that we haven't met yet, right? We should expect them to have something like Bayesian inference, be a good explanation for how it is they solve inductive problems, something like logic giving us a sort of good story about how they might solve deductive problems. But then it's going to become more complicated as we start to go down into those other levels of analysis, where that's where a lot of the complexity arises, right? And where we might not expect that we're going to find simple stories. In fact, it might be a multiplicity of stories, each of which has some simple components. And that's part of, I think, what makes it interesting to be a cognitive scientist.

1:18:35.9 SC: It's definitely the fun place to live intellectually where you do know some things, but there's still an enormous amount of things that you have yet to figure out. So, Tom Griffiths, thanks very much for being on the Mindscape podcast.

1:18:47.9 TG: All right, thank you.

[music]