325 | Alvy Ray Smith on Pixar, Pixels, and the Great Digital Convergence

0:00:00.5 Sean Carroll: Hello, everyone. Welcome to the Mindscape podcast. I'm your host, Sean Carroll. You don't need me to tell you that we live in a digital world these days. We have digital cameras, we talk to each other on digital computers, listen to podcasts, our audio is digital, the whole bit. We've digitized most of human knowledge and are training our AIs on them. We've had a transition from thinking of the world and representing the world and picturing the world in mostly analog ways, ways that involve smooth configurations of matter that you would, in principle, need an infinite amount of information to reproduce, to a digital world where we have finite amounts of data that we can store in a computer, that we can manipulate, compute on, and change the world around us with all that technology. We see the changes going on. We're really just at the beginning of what these changes are going to bring to us. One example of the digitization of the world is the existence of computer animation, right? You can make movies. You can make very popular movies such as those made by Pixar, as well as DreamWorks, Disney Animation, other studios, and they can be just as emotionally compelling and good storytelling as movies made with human actors.

0:01:21.3 SC: You need both. I'm not saying you're gonna replace them. The replacement narrative is a little bit misplaced sometimes, but it's there. It's a form of art, digital media. Today's guest is Alvy Ray Smith, who is a well-known computer scientist, a pioneer in computer visualization and computer art, and he was a co-founder of Pixar. And what we're gonna talk about today is two aspects of this digital revolution, what Alvy calls the Great Digital Convergence. One is once you have the world in digital form, you can do things to it, right? You can manipulate it. You can think about it. You can create in new ways. The other way is, are you worried that maybe you're losing something, like literally losing information about the world by representing it in digital form? It turns out that under a very realistic set of assumptions, you are not. And there are mathematical theorems that show you this, and that's a kind of fun thing for Mindscape listeners. But we're gonna get there historically. We're gonna get there by talking about Alvy's journey from someone who is interested in computers and in art from a very early age in the 1960s in New Mexico, up to thinking about and developing ways to make computer graphics, ways to animate them, a decades-long quest to literally make the first all-digital computer-generated movie, which of course eventually came out as Toy Story in 1995.

0:02:53.8 SC: Alvy's also the author of a book that came out just a couple years ago called A Biography of the Pixel. And in the book, he talks both about the science of turning the world digital and a little bit about some of the personalities, some of the history, up until the present day and up until the recent history of making Pixar and things like that. I just love the idea, the sort of conceptualization of what is happening to us in the modern world as a digitalization and how that changes our capacities, our affordances, our abilities to do things. And it can be very hard to predict what will come next. It's this background that I think is actually very useful sort of for giving us a context in which to think about what comes next in the digital world. And as Alvy says, there are theorems in computer science that say you can't really know what's gonna come next. You have to do the computation, or in this case, you have to live through the history of the future. So that's what we're here to do. Let's go.

[music]

0:04:09.7 SC: Alvy Ray Smith, welcome to the Mindscape Podcast.

0:04:12.3 Alvy%20Ray Smith: Good to be here, Sean. Let's go.

0:04:15.0 SC: You know, usually I really focus the Mindscape Podcast on ideas. And you've had a lot of ideas over your life. You've written this wonderful book, The Biography of the Pixel, that has some great ideas and we're going to get there. But I think this is a case where maybe a little biography would be good to set the stage a little bit. Tell us about how you got into the idea of computer graphics and ultimately bigger ideas that culminate in digital light.

0:04:45.1 AS: I look back at my life and I can see how I feel. I must have been just grabbed by the neck and pushed this way all along.

[laughter]

0:04:51.5 AS: It just happened. I grew up in New Mexico and one of the lucky breaks of my life was I had an artist as an uncle. Uncle George taught me how to oil paint. I could sit on his floor. He was the only relative he would allow in his studio.

0:05:09.1 SC: Oh, wow.

0:05:10.5 AS: As long as I sat on the floor and didn't utter a sound. And I did that. And I learned how to prepare the brushes and mix. You know, I fell in love with the smell of turpentine and linseed oil and all the media aspects of oil painting and learned how to prepare canvas and so forth. So I grew up painting pictures. And I was also smart. So, you know, got promoted to the top of education system. And long story short, when I discovered computers in New Mexico, I fell in love, as many people have. [chuckle]

0:05:48.4 SC: What year are we talking now?

0:05:50.9 AS: Oh, we're talking the '60s.

0:05:52.3 SC: Okay.

0:05:53.1 AS: I was at New Mexico State University for my undergraduate work, and this was '61 through '65, somewhere in there, early '60s.

0:06:02.2 SC: Not a lot of computers hanging around. You could have gone your life...

0:06:04.6 AS: No.

0:06:04.7 SC: Without bumping into computers.

0:06:06.3 AS: This is when you had to submit a deck of cards through a window to a holy priesthood, and if you were lucky, it would run.

[laughter]

0:06:14.7 AS: And, you know, I grew up with rockets going off next to White Sands Missile Range. Wernher von Braun came there after World War II with his V-2s to show American rocket scientists how to do missiles. So I grew up thinking it was normal to see rockets going off over the mountains every day. And I heard the first A-bomb go off. And so I was surrounded by high tech. And I discovered computers and art. You could sort of see how that was gonna...

0:06:41.8 SC: That already there. You're determined. Yeah.

0:06:42.9 AS: It was gonna happen sooner or later, right?

[laughter]

0:06:45.7 AS: It took quite a while to happen, but that's the beginnings.

0:06:49.6 SC: And then...

0:06:49.9 AS: And then I went... Go ahead.

0:06:51.2 SC: Yeah, you went to New York. Is that right?

0:06:54.1 AS: Well, I went to graduate school at Stanford. They had a new department there called computer science. You may have heard of this. Yeah. And they had a subject that I wanted to learn called artificial intelligence.

[laughter]

0:07:09.5 AS: This is 60 years ago, Sean.

0:07:11.5 SC: Yeah.

[laughter]

0:07:12.7 SC: There's been a couple of winters since then.

0:07:15.4 AS: I learned from the original guys. And after a couple of years, I went, "You know Alvy, this is not gonna happen in your lifetime." It been understanding how the human brain, mind works, using the computer as a model. That's what I thought it was, right? And after a couple of years, I could see they weren't anywhere. In fact, I look back at it now, it was kind of the silly level of AI. I even taught a course or two at NYU, which was my first job. And I was working with my chairman, my first boss in academia was a fellow named Herb Freeman. And he was one of the earliest computer graphics people in the world. And I didn't fully appreciate it at the time. I look back now and I'm so embarrassed. I didn't really appreciate it. He knew I was an artist. He knew I was a theoretical computer scientist, proof theorems and all that. He thought I would be a magical, you know, the right mix to join him in research of making pictures on computers. Of course, I was, but I didn't realize who he was.

[laughter]

0:08:24.8 AS: He came to me one day and said, "Why don't you, you know, Alvy, why don't you come over here and start working with me and my guys?" And I said the following, I said, "Herb, if you ever get color, I'm in." Because it was all black and white in those days. And I thought black and white was absolutely boring, right? [laughter]

0:08:41.0 SC: Yeah, I remember those black and white pictures.

0:08:44.5 AS: So I kind of missed four years of event. I could have left even four years there. I mean, as if I wasn't on the leading edge anyhow. But then I went to Xerox PARC. I broke my leg skiing in New Hampshire, and I was lying in a full body cast for three months. And I got, you know, people rethink their lives. You've heard these stories. I rethought my life. And I said, "Alvy, you're making a terrible mistake. You're going, you're advancing right up the academic ladder. No problem. Getting papers published, advancing up the degree, getting fancier and fancier titles, getting grants. But you're not doing anything about your art. That's wrong. What you need to do... " and Sean, I look at this, how could I have been so crystal clear? Because it was content free. I resolved that when I came out of the full body cast, I would resign my professorship and go to California where something good would happen. That was the total content of my thought process, okay?

[laughter]

0:09:53.0 SC: Right.

0:09:53.9 AS: But it did, it did. I did that. I went to California, to Berkeley, as a matter of fact, and my best friend, Dick Shop, had been encouraging me to come to this place where he had just gotten a job called Xerox Palo Alto Research Center, the famous Xerox PARC, in its heyday. And finally, one day, for reasons we don't need to go into, I said, "Okay, I'm coming over. I want to spend the night with you." And he said, "Okay, but you've got to come see what I'm doing at PARC." And what he had done was built and programmed a paint program, color pixels. It was the first color pixels. Until I wrote my book, I thought they were the first color pixels. It turns out not to be the case. But for all practical purposes, they were the first color pixels. And I looked at it, and it was a paint program. I'm a painter. I was looking for color. He was my best friend. Boom. He had me, you know, had me hired on at Xerox PARC really fast to show off, artistically show off, the paint program at Xerox PARC. So, okay, so it's starting to come together, right? Then Xerox fired me.

[laughter]

0:11:07.9 SC: That happens. Yeah.

0:11:08.8 AS: Yeah. I went in and I said, "Well, but why?" And they said, "Well, we've decided not to do color." And I went, "Oh, wait a minute. You own color." I mean, I didn't know there was actually some color pixels earlier, but I thought, and it was pretty close to true, they owned all the color pixels in the world. And I said, "The future's color and you own it all. That doesn't make sense." "Alvy, this is what my boss... Alvy, you may be correct, but it's a corporate decision to go black and white." So I went, "Okay, bye."

[laughter]

0:11:51.2 AS: But I had started working with a friend, an artist friend from San Francisco, a video artist named David DeFrancisco. He would come down and we would jam all night on the paint, super paint. That's what this machine was called. First color pixels. Everybody who saw it just blew their minds 'cause nobody had seen color pixels before ever, right? This was it, except some astronauts that I could go into later.

[chuckle]

0:12:18.4 AS: And he said, "Alvy, we should submit a grant proposal to the National Endowment for the Arts to exploit this new artistic medium." He didn't have a job. I had a job, but he needed a job, and this is how he was thinking.

0:12:31.4 SC: Yeah. He started as an artist.

0:12:32.8 AS: I said, well... Yeah, he was. And I said, "Oh, I hate doing grants. I submitted grants to the National Science Foundation. They had to be submitted in 40 packets, 40 copies of this," and he said, "No, no, no, no, no. That's not the NEA. The NEA is one page, and you submit your work." I went, "Oh, okay." So we submitted. We did a videotape, and we submitted that. And then Xerox fired me. I lost access to the only machine in the world that could satisfy this grant if we got it. So we had to, David and I had to go find the next frame. A memory of pixels is called a framebuffer. It was called a framebuffer in those days. Holds one frame. Okay. At video resolution was the idea. We had to find the next framebuffer. So we jumped in my car and drove over to Salt Lake City because we had heard that this company called Evans & Sutherland in Salt Lake City was building the first framebuffer that you could buy. It was the second framebuffer in the world, and it was a commercial entity. So we get over there. Now, I have to paint the picture a little bit. I've just come out of the late '60s. I've got hair down to my ass. David's got electric hair out and we were out in our two feet.

[chuckle]

0:13:57.8 AS: And we resolved we wouldn't use the word art when we were in Salt Lake City because it was Department of Defense funded. But, you know, we were dead giveaway no matter what we said. So they said, "Folks, guys, we can't take you on here." All we were asking for was access to a framebuffer. That's all we needed. We just needed access in case we got this grant. We can't do that. But a rich man just came through here from Long Island and bought one of everything in sight. And I said, "Including the framebuffer?" And they kind of looked at each other and finally, you know, I don't know what they consulted, but yes, he bought a framebuffer. So this probably won't work, but there's a... See that teapot behind me?

0:14:48.1 SC: I see a teapot. The audience can tell.

0:14:50.2 AS: A teapot is very famous in computer graphics. That teapot in particular is a copy of Martin Newell's teapot, an Englishman who was working at University of Utah. And his teapot, he decided to digitize it. So in those early days, there were no databases that we scientists could use to try our rendering algorithms on to see how they look. Did it work? Did the surfaces actually reflect or whatever we were trying to do? And so, Martin had carefully hand digitized the 3D points on the surface of his teapot. And then he would hand his teapot database around. So it became the holy icon of computer graphics. And everybody who did anything always rendered the teapot.

[laughter]

0:15:37.6 AS: And that is now become, I'm a member of the SIGGRAPH Academy, and that's what you get if you're a member of the SIGGRAPH Academy. You get an actual ceramic version. He says, "I happen to be going out to visit the rich man on the North Shore of Long Island, and I'll call you and tell you what I see when I get back." And he did that. And basically what he said was, if I were you guys, I'd get on the next plane. This guy had a full animation studio on his campus. All right. This is really hard to describe 'cause his place was called the New York Institute of Technology, NYIT. And at the time, it was essentially a diploma mill. It's now turned into a reputable outfit. But then it was basically a diploma mill for kids on Long Island who couldn't get into real colleges.

0:16:29.4 SC: [chuckle] Okay.

0:16:30.2 AS: Yeah, yeah. The rich man's name was Alexander Schure. Nobody's heard of him. He had cobbled together four of the gorgeous estates on the fabulous North Shore of Long Island. This is great Gatsby land. This is just posho beyond belief. And we had all sorts of names. Cootville land, we called it. It was the wealthy. And the buildings on those estates, the mansions, were the buildings on his campus. So we did video in one mansion. We did video graphics in another mansion. I lived in another mansion. David and my girlfriend lived in another mansion. It was just all mansions. I'd get up in the morning pinching myself. We talked our way in, is what I'm trying to say. I did. Took David a year to get in. But I talked my way into this crazy man, Alexander Schure, who had an animation studio of the old-fashioned ink and paint type of animation, cell animation. And he basically wanted to be the next Disney. And he thought the computer would be the way of making money. He was really an entrepreneur. He wanted to get rich.

0:17:43.4 SC: What year are we in by now?

0:17:48.1 AS: This would be the mid '70s.

0:17:49.6 SC: '70s. Yeah, okay.

0:17:51.4 AS: Yeah. Yeah, I think I've got that right. Yeah, 'cause I exited in '79. And we were in heaven. And he bought us everything in sight. So one day he came to me. This man is really hard to get across to people 'cause he didn't talk like you and I are talking. Right now we exchange, and there's sort of a rhythm to a conversation. He just spewed word salad, we call it. Casey Stengel's speech. He just talked. It was just stream of consciousness. And I remember at first I didn't know what to do, so I just started talking. And after a while I noticed that if I heard my words come out, you know, worked into his stream, I must have made, we must have communicated.

0:18:42.8 SC: Success, yeah.

0:18:43.7 AS: So I'm not going to go through the full details, but that was the technique. So he asked me, using this odd technique one day, he said, "Do we have the best computer graphics facility in the world?" I said, "Yes, we do," 'cause we did. And he said, "How do we stay there?" Which was the right question to ask. And I said, "Well, if you buy me... " Remember I mentioned the framebuffer was the reason I was there, was this piece of equipment that held pixels. It was eight bits deep, 256 colors, basically 500 by 500 pixel resolution, which was video at the time, $80,000. $80,000, $1975 for eight bits. I said, you know, brash kid that I was, I said, "Well, if you buy me two more of those frame buffers, those eight bit frame buffers, I can stack them up into a 24 bit pixel memory. And then we have 16 million colors and then we can do anything." And he wanders off. I'm not sure this has gone through at all, right? 'Cause it's just no idea. Well, several weeks later, he comes back. He said, "Well, Alvy, I got you five more of those eight bit thingies. So you'd have two of those 24 bit thingies." You know, and I didn't know until I wrote my book and kind of sat down and ran the figures that just on my say so in that weird conversation, he had spent $2 million current money on me.

0:20:14.9 SC: Wow. Yeah.

0:20:15.7 AS: Just boom, just like that. So we had the first 24 bit pixels, which is what everybody uses now in the world. And we went bananas. Artists especially just started creating art just nonstop. And, you know, it was heaven. We were in mansions. We were gorgeous. It was heaven. But it really wasn't going anywhere 'cause we finally figured out this guy. Oh, by the way, the group that had assembled there included Ed Catmull, who would become the co-founder of Pixar with me.

0:20:50.9 SC: Right.

0:20:52.4 AS: Or a few people from University of Utah were there. We decided, you know, what we should do, we should be the first group in the world to make a movie, complete digital movie. Let's be the first. Now, it took us 20 years to do that...

[laughter]

0:21:08.6 AS: Which we didn't realize it would.

0:21:09.7 SC: But you did it.

0:21:11.8 AS: We did it, but it took 20 years and a lot of ups and downs in between. But that's where the idea started. But let's see. So Alex, Uncle Alex, we called him for a strange reason, was, like I said, he had this animation team on his campus. They were creating Tubby the Tuba in the old fashioned way. And he was hoping, in his words, he was saying, "Can you make Tubby dance using the computers?" We couldn't. We could hardly do anything because computers were so slow still. Eventually, the movie came out. And I remember sitting in a screening room in Manhattan. We were all [laughter] horrified by this movie because anything that could be wrong about a cell animated movie was wrong. There were shadows under the ink lines. There were dust on the frames. The story sucked. It was just everything was bad. One of the animators, the young animators, whom we'd befriended, came up and said, "I've just wasted two years of my life," and he was gone. So we knew that this was not the guy that was going to get us to the first complete movie, right? He just didn't...

0:22:29.7 SC: But sorry, you were clearly interested in some intersection of art and computers very early, but was it his, uncle... What was the name? Uncle...

0:22:40.8 AS: Uncle Alex Schure, Alexander Schure.

0:22:42.2 SC: Uncle Alex, was it his initiative that turned the specific form of art toward video, toward movies?

0:22:51.9 AS: I guess you could put it that way. I mean, it never was that clear.

0:22:55.7 SC: Explicit. Right, yeah.

0:22:57.4 AS: He was making a movie, and he wanted us to help him make that movie. And the result of that was we were surrounded by an honest-to-God, Hollywood-educated film production crew who knew how to do it. And they showed us all the nitty-gritty of making an honest-to-God cell-animated movie, so well that we were able to digitize that process for Disney years later. That's one of the things we did to make the relationship with Disney work. So in a way, you could say that, although we kind of knew that... I don't know. I look back, and we certainly didn't conceive of a Pixar-type movie yet. We would have been happy with a 2D movie, anything.

0:23:43.7 SC: Yeah. A Disney-type movie, yeah.

0:23:45.8 AS: As long as it's completely done on a computer. So drawing, he was still using hand-drawn animators who were still drawing pencil on paper, and then we would digitize that. Nope, that's disallowed. You've got to do everything on the computer. Okay, that's the rule for this particular [chuckle] project. Anyhow, about that time, we were realizing this guy hasn't got it, even though this is heaven on earth. We were just treated like gods, and it was wonderful. George Lucas knocked on the door, and Francis Ford Coppola. Two separate days. Not themselves, their people.

0:24:30.3 SC: Sure, sure.

0:24:31.5 AS: One day, Coppola knocked on my door, his people knocked on mine. I knew the guy who was representing Coppola. I distrusted him. And when I realized, if this is who Coppola has, this won't work. "Cause we're going to cost a lot of money. Somebody's gonna have to be strong and steady on the finance part for a lot of years, 'cause this is a biggie. And even though I think Coppola is by far one of the greatest directors of all times, he was already famously impractical about money.

0:25:06.5 SC: Yeah.

0:25:06.7 AS: He would come, he would... Okay. Francis and I remained friends over the years, but that's a separate story. George, on the other hand, we had seen his movie. We had rolled in. Again, I jumped in my car, rolled into Manhattan and seen this movie, Star Wars, which blew us away like it did everybody. And so, we always thought it was gonna be Disney that finally came and did us, paid for us. And Ed and I would make secret pilgrimages to Disney every year.

[laughter]

0:25:41.8 AS: We were so sure that it was Disney, we would just make sure they understood where we were and what we were doing and what we could do for them. It never worked because Disney was run by a football player during this period. His name was Ron Miller and he was an LA Ram. And his qualification was he had married Walt Disney's daughter.

0:26:03.8 SC: There you go.

0:26:05.1 AS: And basically, they would reissue Cinderella or Pinocchio every year or so, but the company was just... But on these visits that Ed and I would make, we got to know the underlying technical team. They knew what we had. They knew that Disney needed us. They also knew that we were going to just hit a dead end as we went up to the VPs. And we did. They just wouldn't listen to us. But that's when I met Frank Thomas and Ollie Johnson and some of the great old animators. We just learned the ropes. So, much to our surprise, it wasn't Disney who knocked. It was George Lucas's real estate man, it turns out, knocked on our door. He said, "George is looking to digitize Hollywood." Well, that's a noble ambition and we thought that was the right thing to do and we could do that. The big mistake we made at this time was to think that he wanted us in his movies. He wanted us to make pictures in his movies. We just assumed it, 'cause we were about making movies, right? I already had a piece in Museum of Modern Art in New York.

[laughter]

0:27:22.7 AS: Content was what we were about. Yeah, we can program and build hardware. Yeah, we're absolutely willing to do that to earn our keep. So, got hired on at Lucasfilm and I started hiring the best computer graphics people in the world. Everybody assumed that we would be in the movies. Everybody wanted to be in the movies. It's just kind of a natural instinct, right? So I'd get these people who are just the creme de la creme in the technical aspects. And also animators started showing up wanting to be part of it. So like Brad Bird was one of the first to show up. And he was the funniest guy I'd ever met. Just had me in stitches every moment. He was just nonstop. He was one of those naturally funny people. But he was also a sensitive artist. And in those days, we were Lucasfilm, but we were in a converted laundromat. And it was freezing cold because the machines had to be heavily air conditioned. They broke all the time. They were slow as all get out. We would put up with it because we understood Moore's law. We were going to be saved by Moore's law, basically. But I just couldn't do that to Brad. I said, "You know, we can't do that yet. You don't want to be here. Not yet." So the animators are starting to get the idea that this new thing was coming along and they wanted to be in there. And so, I got this team together and waited for George to come around to ask us to be in his next movie, George Lucas. He never came.

[laughter]

0:29:11.5 AS: And all of a sudden I went, "Oh, my God, he doesn't get it. He doesn't know what he has. He's got the hottest team in the world and he doesn't know it."

0:29:22.1 SC: What did he want from you?

0:29:24.9 AS: Hardware and software.

0:29:25.8 SC: Hardware and software?

0:29:26.1 AS: To digitize. He wanted us to digitize audio production, video production, and keeping track of the books. And he also wanted us to... Do you know what an optical printer is? I think that's the machine that loses most people. Optical printer mixes film from two different cameras together into one piece of film. It was an art form, basically, at the time. And he wanted us to digitize that process, which we did, by the way. We did all of this. We came up with an audio editing system, a video editing system. We did logistics. We did a digital optical printer. We did all that. But what we really wanted to do was make content.

0:30:11.8 SC: And you knew at the time about Moore's Law, and you could sort of sketch out what the requirements would be to do what you wanted to do, so you could sort of tell eventually you'd be able to get there even though we can't do it now.

0:30:24.7 AS: Yeah, it's just everybody knew that computers are getting faster all the time. It was just part of the structure of the universe. People already knew it. I can't remember whether we called it Moore's Law yet or not, probably. I imagine we did, frankly. But we just knew that every year was better than it was last year. You needed somebody wealthy behind you who would keep buying the latest of everything. So here was George Lucas, and he was apparently doing that. He set us up really well. He let me hire all these amazing people. So what saved our bacon this time was Star Trek II. So Paramount came to Industrial Light & Magic, which is the special effects branch of Lucasfilm, and asked for special effects for Star Trek II, The Wrath of Khan, their movie, the Star Trek franchise. They asked ILM if they could do some of this newfangled computer graphics stuff. And ILM said, "Well, no, we don't do that. I think the new guy's next door," that would be us, "I think that's what they do." So they called me over, and they sketched out this scene requirement they had. And I listened to it for a while, and I kind of sat back finally and said, "Do you guys have any idea what we can do this day and age with a computer?" "No, no." They had no clue.

[laughter]

0:32:10.4 AS: And I could tell from what they proposed, they had no idea. I said, "Okay, I'm gonna go home tonight and draw a storyboard and come back tomorrow and present it for something we can actually do now, utilizing all the best talents that I have." You know, I had a guy on fractal, a guy on bump mapping, a guy on texture mapping. You know, I had these great rock stars. [chuckle] So that's what I did up all night, walked back in with this hand-drawn storyboard and got the job for a 60-second shot. We call it the Genesis demo.

0:32:50.2 SC: I remember the scene.

0:32:51.6 AS: Oh, okay, okay. At the time, it was a mind-blower. It looks kind of crude now, but at the time, it was a mind-blower. We were extremely, extremely proud of it. So I pulled the team together and I said, "Look, this is it. This is our big break. This is the first time we'll be on the big screen in part of a movie that's going to make a lot of money." There had been computer graphics in movies before, but they failed. Looker. You ever heard of Looker? Probably not.

0:33:26.8 SC: Nope.

0:33:26.9 AS: You have?

[laughter]

0:33:33.2 AS: So what we're gonna... And this is a 60-second piece, so we're gonna make Paramount really happy. We'll execute the shot as promised. It'll make the Trekkies in the audience really happy because it'll be exciting. You know, fit the narrative. It'll make perfect narrative sense. Everything here will make narrative sense. But what this really is is a 60-second commercial to George Lucas so he'll know what the hell he's got. And I knew one thing about George, and I look back at it now, I still don't quite know why I knew this, but I did. Probably 'cause I'd been watching movies with him and I just picked it up. He always, always is aware of the camera. Now, if you go to the movie and try to stay aware of the camera, then the director has failed because you have not been sucked into the emotion of the piece. And apparently, George could do both, 'cause, you know. But I knew he knew where that camera was. So I said, "We're gonna put a camera shot in here. Again, it won't be gratuitous. It'll make perfect narrative sense, but it'll be something that no real camera could possibly do and it'll blow his socks off." And that's what we did. We put a very elaborate...

0:35:04.5 AS: So the camera's attached to a speeding satellite that has to turn around and look at this planet, and it has to track the flame coming over the limb of the planet, and it has to swoop around and do all sorts of stuff. Now, it's doing complete 3D moves that no real camera could possibly do. That's what we knew George would see, or I assumed he would see. All right. The day after the premiere, George, who had never spoken to me, hardly at all, I think he was shy, stepped one foot in my office and he said, "Great camera move."

[laughter]

0:35:46.3 SC: You'd read him correctly.

0:35:47.8 AS: He got it, and he had us in his next movie, just a tiny piece, which was Return of the Jedi. And more importantly, he told his best friend Steven Spielberg about us, and Steven had us in his next movie, Young Sherlock Holmes, and then the word started to spread. But we didn't want to be effects animators. We wanted to be character animators. We wanted the characters to come alive, just as Walt Disney had taught us on his TV show when we were little kids. He showed us how to do it. Well, he showed us how they did it.

[chuckle]

0:36:26.4 AS: That's what we wanted.

0:36:26.5 SC: But let me just say, that's a non-trivial statement about you and the people you were working with. Plenty of people would have been just happy being special effects animators, but you wanted to...

0:36:36.7 AS: And there were, a lot of companies were.

0:36:37.9 SC: Yeah.

0:36:38.0 AS: Yeah.

0:36:38.5 SC: You wanted to make characters and tell stories.

0:36:42.1 AS: Yeah, yeah. 'Cause we had been, you know, I grew up watching, you know, Cinderella, Pinocchio, and just been in love with these wonderful movies. Took you out into a completely different world. And then Disney himself, old Walt, showed us how to do it. You know, draw and ink and paint and so forth. But then George and his wife, Marcia, got divorced.

0:37:12.8 SC: All right.

0:37:13.0 AS: And in California, community property state, overnight, George loses half his fortune. And we're an expensive group. So I went into Ed, my partner, all these years, and I said, "Ed, we're gonna get fired. George never really understood who we really are. We cost a lot of money. He's just lost half of everything. He's gonna fire us." And it would be, and I use this word sin because Ed was a good Mormon and I grew up Southern Baptist, so we know these words, right?

[laughter]

0:37:50.5 AS: I said, "It'd be a sin for this world-class group to disperse." And I should add at this point that we knew, we had a cold fact under our belts at this point, that it was gonna take five more years to get Moore's Law to where it needed to be to make a movie. So we had to hold this group together for five years somehow. And it can't be computer graphics 'cause that's not here yet. "Let's start a company," I said. He agreed. Now this is two nerds talking to each other, right? We're not.

0:38:26.3 SC: [chuckle] Not captains of industry.

0:38:29.1 AS: No, not at all. We'd always been working for the man.

0:38:31.9 SC: Yeah.

0:38:32.1 AS: [chuckle] And so, we went across the street in Marin County and to a bookstore called A Clean, Well-Lighted Place for Books, I remember it well.

0:38:45.4 SC: Oh yeah.

0:38:46.3 AS: In Larkspur Landing, Marin County. I bought two books. Ed bought two books. How to start company books, how to read profit and loss sheets, how to do business plan, you know, [chuckle] really basic stuff. We had no idea what we were doing. But the damnedest thing is it worked. We wrote a business plan. We decided, "Okay, the only thing we can do to support a group of our size, which is about 40 people now, is hardware." Doing software, doing commercials didn't pay. Doing software didn't pay. The only thing that could possibly pay for this is hardware. Well, we had built, I called it a digital equivalent of an optical printer earlier for George. We had built this special purpose computer. And so, I said, "We could just take that prototype like they do in Silicon Valley and turn it into a product and sell that. We already have the prototype." Then we've got the team that did it. So it's just sitting there. So that's what we decided to do, write up the business plan. And then our troubles began because you've got to find capital when you start a company, right? [laughter]

0:40:11.4 SC: Yeah.

0:40:12.6 AS: The hardest part. Well, Ed and I talked to every venture capital firm in Silicon Valley. Sand Hill Road's famous for being the folks. Okay, so we were lucky in that we were sexy. We were part of Lucasfilm. So we got in the door, whereas a lot of entrepreneurs can't even get past the door, right? Well, we got past the door, but 35 of these outfits turned us down. We talked to everybody. They turned us down. We were in the wrongs. They didn't. We were presenting ourselves as a hardware company with this, but we had no experience. [chuckle] We already had the prototype built. We were past seed money. We didn't fit, 40 people. It just didn't fit. And they said so. It's interesting. Good luck. So the next idea was we would form a strategic partnership with a large corporation. So we started talking to large corporations, and we talked to eight who all said, not a chance.

[chuckle]

0:41:29.7 AS: And then two said, okay. General Motors, the largest corporation in the world at the time, and Philips of the Netherlands decided to go together. If we would do renderings of automobiles for GM and medical imaging for Philips, they would finance our company. That almost happened. That got right to the final stages of negotiation in a Manhattan skyscraper right above Grand Central, I remember. It's one of the most incredible days of my life. Four parties, 20 attorneys, just log jams, stress headaches. It's just an awful day. But at the end of the day, everybody's smiling and shaking hands. And in the business world, that's a done deal. But not this time. I mean, usually overnight, the attorneys write up what just got said.

0:42:41.4 SC: Yes. Write up the details.

0:42:42.9 AS: And there's a signing. Okay, it's done. No, not this time. Because at the same time we were doing this meeting, H. Ross Perot, remember this guy?

0:42:52.5 SC: I will know, yeah.

0:42:53.3 AS: H. Ross Perot, it was his branch of General Motors we were dealing with. And he was uptown insulting the board of directors of General Motors at the GM building.

[laughter]

0:43:04.9 AS: And overnight, it broke that basically they had split. GM was getting rid of H. Ross Perot. Well, we were the baby in the bathwater. That was us. It was dead overnight. So we're now 45 deals. Ed and I have tried 45 different ways of getting financed. That's a lot. And everybody's saying, no, we thought we had it and didn't. And so Ed and I are in the limo going from Manhattan back out to JFK to fly home. And we're just frantic, you know, now what do we do? When we came up with what I call our Hail Mary pass, neither one of us being sports guys, by the way.

[chuckle]

0:43:57.6 AS: Yeah, no. "Let's call Steve Jobs." Now, Steve, we had already met Steve. He had been fired from Apple. And he had called Ed and me down to his mansion outside of Woodside, California, which is near Palo Alto. And I remember sitting on the grass and he proposed to us that he'd buy us from Lucasfilm and run us as his next company. And we went, "No, we want to run our own company, but we'll take your money." And he agreed. Okay? He agreed to that. So, but every deal had to be cleared through Lucasfilm 'cause they were getting a piece of the action. Well, Lucasfilm thought they had General Motors, Phillips on the hook at this wonderful valuation, which turned out to be about twice the valuation of what Steve was offering. So they kind of more or less laughed him out of the office, so to speak. All right, but at this point, GM, Phillips has failed. Ed and I say, "Let's call up Steve and say, Steve, everybody's at the end of their rope. Just call Lucasfilm and make exactly the same offer again. Same valuation being half of what, okay. Otherwise, nobody's getting anything out of this, right? Lucasfilm loses everything and we lose our company." And he did. And that's how we got [laughter] financed, by a hardware guy, we thought, who could help us as a young hardware company start this company.

0:45:39.1 SC: Were you at the time still thinking in the back of your minds that you're presenting yourselves as a hardware company, but you're gonna keep alive the dream of making a movie?

0:45:47.6 AS: Oh yeah, we made it clear. There was always be this group. We had John Lasseter already, the animator. We had the guy, we had our core team there. Everybody was aware of it, that that's what we really wanted to do. But in the meantime, to make money, to pay for the corporation, hardware, marketing, distribution, all that. Well, and we had this hardware guy, we thought, on the board. So Ed and I and Steve Jobs are the board. And we failed. As a hardware company, even with Steve Jobs, we failed. I think three times, depends on how you count or measure. But when you run out of money and you can't pay the bills in Silicon Valley, you're dead. Except Steve did not want to be embarrassed by failure after having been booted from Apple. So he would tear into me a new one, but he would write another check. He would take away equity from us, from the employees. Originally, the employees owned 30% of the company. So he would take that away, pieces of that away and write a check. And he did that three times, I think, until he had taken away all equity. He owned it all. He finally did buy.

0:47:06.4 AS: A lot of people say he bought it. No, he didn't buy it. He did buy it eventually from the employees over a course of several years. And we didn't have anything, but we had our company. We kept it together. And we kept it together for five years. Moore's Law did its thing, and then almost right on cue, Walt Disney shows up. Now, we had already made a great relationship with Disney with a project called CAPS. But they said, "Let's make that movie you guys always wanted to make. We'll pay for it. You do it."

0:47:44.6 SC: So what year are we now?

0:47:47.1 AS: This has got to be like '84, maybe '85, somewhere in there.

0:47:50.2 SC: Oh, okay.

0:47:51.7 AS: No, wait a minute, I'm wrong.

0:47:53.2 SC: That's pretty early.

0:47:54.5 AS: Pixar started in '86, so this must be '91, 1991.

0:47:58.3 SC: '91? Okay, that makes more sense. Yeah, good.

0:48:00.2 AS: Yeah, yeah. Okay, 'cause it took several years to actually execute the movie once they started funding it. Okay, and the movie came out in '95, I think. So Disney saved Jobs' financial ass 'cause they came in and paid the bills. They also knew how to make movies.

0:48:25.1 SC: Yeah, they actually wanted you to make a movie.

0:48:27.7 AS: Yeah. They actually wanted us to do what we always wanted to do. And Steve, to show off his business smarts, which he does have a lot of, decided, okay, so Toy Story gets made, it's in the can. Pixar takes it to Manhattan and the critics see it, and they go nuts. Nobody's seen anything like Toy Story before, right? Steve's brilliant move was at that point to take Pixar public based on nothing but having that Toy Story in the can. We had no money.

[laughter]

0:49:12.9 AS: But it was a brilliant bet, and he became a billionaire overnight from that bet.

0:49:19.6 SC: Right.

0:49:19.9 AS: Okay.

0:49:21.5 SC: And when you say, you know, you were thinking since the seventies about making a fully digital movie, the movie that was eventually made was Toy Story and it was obviously an enormous success, but you weren't thinking of Toy Story in the 1970s. You just wanted a movie. [laughter]

0:49:36.7 AS: Not at all, no. We had no idea that we were going to be Pixar or that that would be our look, you know.

0:49:45.3 SC: Or the plot or the characters, you know, all that.

0:49:48.3 AS: None of that. In fact, most of that came out of John Lasseter's head. He was this amazing creative animator that I hired along the way. Again, it was one of those lucky things that happened. I told you about how we visited, Ed and I would go down to Disney like once a year. Even while we were at Lucasfilm, we're still going down to Disney, all to keep that relationship alive. Somehow, they've got to figure in somewhere. And I'm leaving out a whole CAPS project, but on one of these visits, a young animator named John Lasseter is there. He's working for Disney. He's really excited to meet us and he says, "Hey guys, you want to go down into the archives?" And I says, "Yeah. Yes."

[laughter]

0:50:38.8 AS: So he takes us down to the archives and says, "Well, what do you want to see?" And I said, "Anything?" And he says, "Alvy, anything. Yes." I said, "Well, I want to see Preston Blair's Dancing Hippos from Fantasia."

0:50:50.0 SC: Okay. [laughter]

0:50:50.7 AS: He says, "Okay." And he goes and looks up on a chart and pretty soon he's got this folder open and he's thumbing through Preston Blair's original drawings of, I think her name was Pintura the Hippo or something like that, dancing. And I'm just flipping out, right? And we're bonding, right? So there's some serious bonding going on at this point. Then he said, "Okay." John says, "Now what?" I said, "Well, anything?" He said, Alvy, yes, anything." I said, "Okay. The drunk elephant scene from Dumbo." Boom, we get that. We're off and running. But we couldn't touch him 'cause he was working for Disney. So a few months later, Ed's down on the Queen Mary, which is the original Queen Mary, which is docked in Long Beach, permanently docked, and it's a convention center now. And he's at some meeting down there and we're doing our daily business talk and he said, "Oh, by the way, John Lasseter just came by and he's no longer at Disney." I said, "Ed, Ed, get off the phone now and go hire him," which he did, of course.

0:51:58.7 SC: And the rest is history. Yes.

0:52:00.6 AS: So he came up and saved. We had decided to do a piece at the 1984 SIGGRAPH to show people we did character animation and not effects. And I assumed I was gonna be the animator, foolishly thought that. And I had done the storyboard and was executing away on it. And then John, we get John. John looks at my storyboard. He says, "Mind if I make a few suggestions?" I said, "No, no, that's why you're here."

[laughter]

0:52:32.8 AS: Well, he softened the character and he made it work. He saved myself. He saved me there on that one. And we took it. This is all happening at Lucasfilm. We took it to SIGGRAPH. It blew SIGGRAPH away. Just 5,000 screaming fans who knew they were seeing the future. Because this was the first glimpse at the so-called Pixar look, right? And George Lucas was in town and he came. This was Minneapolis. And he came because Linda Ronstadt was in town. And they were a thing. So Ed and I and Linda Ronstadt and George all went out to dinner. Then after dinner, we got into a limo and we went down into the bowels of the sports stadium in Minneapolis where SIGGRAPH was being held. SIGGRAPH's the annual computer graphics conference. I think about 40,000 people are attending by now. Huge thing. And the film show had like 5,000 people in the audience each night. And we knew when we showed this piece to this audience, it was going to blow their minds.

0:53:46.7 SC: And George is going to see it. Finally, he would see it. He was expressing interest in what we were doing. And he would see it recognized by our peers in an astounding way. But, you know, he didn't. He saw exactly what he saw. He couldn't induct to what this meant for the future. There was an unfinished, one scene was unfinished. It still had line drawings in it. And, you know, I look at the piece now, and yeah, it's kind of crude a little bit. And luckily, he didn't say so. He hated it. We found out later. He just hated it, but he didn't say so.

0:54:30.1 AS: He would have broken our hearts if he had said so. And, you know, again, why people didn't... One of the fun things, and you look back at this history, is they let the people with the... Steve Jobs, by this time, probably owns nearly everything. And at Pixar, he never got rid of the team that was John Lasseter's team, that was making short subjects. He kept it on, even though he had no concept. It was not his goal to make a movie. He didn't know anything about movies. But he could have just axed it, but he didn't. I guess he knew that if he axed, we were out of there 'cause that was our whole reason, right? Yeah.

0:55:16.7 SC: [laughter] I mean, maybe this is a good time to step back a little bit. And in the book that you wrote, you sort of ended the book with what we've just been talking about, these stories of how you actually implemented these ideas. But you start the book with a very big picture perspective on the digital convergence that we're coming in.

0:55:42.5 AS: Yeah, I would love to talk about this because this is what my book is really about.

0:55:48.8 SC: Well, that's what I really want to hear about. I mean, you certainly convinced us that you've been through the wars here and you're the one to think about this in a careful way. And so, I mean, how are you thinking about... Well, it's your choice. We could talk about the big picture of the digital convergence or we can spend a little fun time talking about Fourier transforms and the sampling theorem. [laughter]

0:56:11.8 AS: I would rather talk about the digital convergence.

0:56:13.9 SC: Good, let's do that.

0:56:14.5 AS: And if there's any more time, we can talk about the sampling theorem.

0:56:17.4 SC: Great.

0:56:18.2 AS: Which is a beautiful thing that I love to present. But what the book is about, and I said so right from the beginning, I think, is that something happened about the year 2000. It was huge and nobody paid any attention. I call it the Great Digital Convergence. So I wrote the book to celebrate this major change in our ordinary life and celebrate it because it was big. Now, the way I've presented it in the book was it's the moment when all old media types disappeared and got replaced by the bit in the form of the pixel. No more film, no more videotape, no more analog anything, no more analog audio recordings. It all became digital. All AV, all audio visuals became video. But Allison, my wife Allison, has helped me put this in larger perspective. She says, "You know, Alvy, it's even bigger than you said in the book." Everything became bits. Everything. The whole world of knowledge became bits. And moreso...

0:57:35.5 SC: Sorry, as opposed to analog. I mean, you're drawing a distinction.

0:57:38.8 AS: Yeah. Well, yes, because, for example, libraries became bits instead of printed pages. All. I mean, there's still things that haven't been digitized, but essentially we now have access to everything or could if somebody just gets around to digitizing it. All right. And Moore's law is still ripping away. This amazing supernova explosion of computational power. The digital convergence meets the supernova of Moore's law and gives us what? Well, we're in this AI revolution that's just blowing everybody away right now. That's one of the results. Now that computers are fast enough to go looking at every piece of fact that's ever been made, they can find correlations that we never suspected were sitting there all along. So ChatGPT-wise. Okay, okay.

0:58:41.8 SC: So, it's a story of turning artifacts into information in some sense. Like I think in the book you mentioned, you know, a painting used to be a painting. Like you had paint and you had canvas.

0:58:55.2 AS: Yeah, the medium used to be important. And all of a sudden we said, "No, you don't need the medium. The picture is independent of the medium. The information is independent of the medium." What a shocker. But that's the way we're writing right now. And I'm continually blown away by it every single day as people come to me and tell me what they can do with an AI program. Like I have a friend in the financial business who says, "Oh, I don't write code anymore, Alvy. I just... The AI does it." And I says, "You mean good code?" He said, "Oh yeah, difficult code, elaborate code, and it's correct." I said, "So you mean all the software guys at your company are going to go away?" And he says, "Yeah, they're not needed anymore. This does a better job." Okay. Now I get stories like that. And I heard about that. It's even in the end of my book. Because while Alice and I were in Cambridge, one of my colleagues, John Bronskill, who made his initial money from Photoshop filters. In other words, he was a pixel packer like I am.

[chuckle]

1:00:20.8 AS: He was going for his DPhil at Cambridge when we were visiting, and he came rushing across when he saw me and said, "Hey, Alvy, we don't need to program anymore." Now this is 2015 or so.

1:00:32.7 SC: Okay.

1:00:33.7 AS: All right? Kind of 10 years or so ago? Yeah. More. Somewhere in there. I think it's more than 2015. Okay. Anyhow, somewhere in there he says, "We don't need to program anymore." And I said, "What do you mean?" He says, "Read this." And he slipped me what I call the Zebra's Horses paper, which comes from Berkeley right here. The AI guys here at Berkeley, who showed that if you trained a neural net in a certain way, you could give it a picture of zebras and it would give you back that same picture where every zebra had been replaced with a horse, or vice versa. And I went, "Wait a minute, that's an ill-posed problem, isn't it? I mean, what's a horse to a computer?" He says, "Alvy, that's what I'm trying to tell you. We don't know how it works. We don't know how it works, it just does." And it's a point where I went, oh my gosh, I always thought I would understand, as we got closer and closer to so-called intelligence, that I would always be able to understand each advance. But of course, Turing told us in 1936, "Oh no, you won't. We can't even tell by looking at a program whether it's gonna stop or not. That's been known for decades." And to think you know where a program's going to go, no. I'm telling you right now, mathematically, you can't know. That's where we are, I think.

1:02:04.2 SC: I mean, it's a deterministic process, but I think, again, as you say in the book, the only way to know what the outcome of the computation is going to be is to do the computation.

1:02:14.0 AS: That's right.

1:02:14.5 SC: In general.

1:02:15.9 AS: Each step is deterministic, but the outcome is not determined.

1:02:20.7 SC: It's interesting.

1:02:21.3 AS: I mean, it's not predetermined, that's what I should say. It's not predetermined.

1:02:23.8 SC: Yeah. I mean, you're married to a philosopher, so you know this careful distinctions. It is...

1:02:29.1 AS: Yeah, oh, absolutely.

1:02:29.5 SC: Deterministic, but you can't know it. So it's kind of, who cares if it's deterministic or not? You got to actually do it.

1:02:35.3 AS: Right, exactly. Yeah.

1:02:36.1 SC: Yeah.

1:02:36.7 AS: So what's that mean? That means you just jump on the wave and write it, 'cause see what happens. Well, I've been doing that all my life, right? I've been writing the Moore's law. I was there 1965 when I came to Stanford, Moore's law factor was one. That was the year that Gordon Moore's explained his observations, thinking it would last for 10 years. Well, it's lasted all my life, and I don't see any end in sight. The one dimension I didn't pursue in my book, and I would have in a rewrite is, oh, by the way, once these computers get very tiny because of Moore's law, you can also explode how many computers there are working in parallel. So that's sort of the Nvidia explosion. It's yet another dimension that makes the exponentiality even bigger than Moore's law, but once you get into exponentials, it's all mind-blowingly beyond human beings.

1:03:37.8 SC: You have this in the book, your version of Moore's law, your way of stating it is kind of a charmingly informal one where you basically say, "Everything good that a computer can do increases by an order of magnitude every five years."

1:03:50.5 AS: Yeah, an order of magnitude is being what a human's capable of. That's about as far as we're capable of seeing into the future is one order of magnitude. I mean, let's take three orders of magnitude, a factor of a thousand. Well, I can tell you how much faster everything will be when it's a thousand times faster, but I can't tell you how different the world's going to be. I couldn't tell you a factor of a thousand ago that it would feel like AI does now. Nobody saw this coming.

1:04:24.6 SC: So I guess that's going to complicate my next question, which is if you think that we are through this Great Digital Convergence 25 years ago or so, I mean, that's still young. That's not that long.

1:04:36.3 AS: No, it just happened. It just happened.

1:04:37.7 SC: It just happened. So what are the implications of this? How do you think we should be capitalizing it?

1:04:42.8 AS: Well, that's why I bring in Turing. He says, "You can't know. You can't know." You can try your damnedest to try to push these, I don't even know what to call them. I don't like saying AI 'cause it sounds like there's a machine that's intelligent and it's just not facade.

1:05:06.9 SC: It's a bad terminology. Yeah, right.

1:05:08.9 AS: It's bad terminology. It's a communication system that, you know. Like Allison points out, the library at the University of California holds tremendous amount of information, tremendous. But you never think of the Library of California as being intelligent.

1:05:27.9 SC: [laughter] No.

1:05:30.3 AS: It's just, okay, it's just, all right. Or the card catalog, which lets you access it, you don't think of it as intelligent either. Anyhow, here are the things that I'm having trouble foreseeing. I don't dare try to forecast what's happening next because I don't think we can know, nor what it's going to feel like, because it sure feels different now than it did back then. And even though we knew Moore's Law would get us here, we didn't know it would feel like this. I keep looking at what it is that made Pixar special, and that's the creativity of the technical geniuses and of the artistic geniuses. And I can't see... I want to believe that machines can get there, but that's religion. I just don't think there's... There's nobody show me any possible way of getting to the creative part. It may just be my desire to hold on to something special for humanity.

1:06:48.4 SC: But I mean, you were there...

1:06:48.8 AS: I'll hold on to it as long as I can. And Sean, I think I'm a machine. I don't see any reason I'm not a machine. So if I'm a machine, there must be an explanation, right? But the levels we're at now aren't the explanation. Okay.

1:07:06.9 SC: The way I think about it is the large language models we've had such enormous success with, or Deep Learning, machine learning more generally, is trained on things that are the products of human creativity. And they're very good at remixing and remembering and distilling and searching and all those things. There's no reason they can't be good. There's no reason why computers can't be good at being creative, but that's not the things that we built right now.

1:07:39.1 AS: Well, also, definition. It's a matter of definition. In my book, I often solve the historical problems by just defining terms. I couldn't believe that I didn't know who built the first computers when I sat down to write the book. That was the word computer. If it had vacuum tubes in it, it was an electronic computer. I went, "No, it's not. That's not a computer." Once I define the terms, then you can just pick off who did what first. It was easy. But you had to define the terms. Same with movies. Same with intelligence. Somebody give me a definition. Just saying intelligence doesn't say anything.

1:08:14.8 SC: In your experience, you've been around a lot of technically creative people, but also artistically creative people. Are these different things that we've just happened to give the same name to, or is there a relationship there?

1:08:25.7 AS: Well, they're different people, that's for sure. I think it's the same thing, but it's applied differently. Probably takes a lifetime of preparation to make the artistic leaps and a lifetime of preparation to make the technical leaps. One of my proudest achievements at Pixar is a culture that honored both of them. Because it would not work if one side looked down their noses at the other side. And by the way, most places I've been, that's the case. Like Microsoft, I worked at Microsoft, heaven on earth for programmers, but they would look down their noses at people who couldn't program. Oh, if you can't program, then maybe you can do this marketing or art or that kind of stuff.

[laughter]

1:09:14.8 AS: In graphic arts houses, just the opposite. You're a creative or a technoid. No, no, no, no, no, no, no. We never let that happen. So everybody, not only was it equal salaries, vacations, titles, all of that, but dignity. You had to appreciate what the other person was doing because the whole company depended on it, that kind of appreciation. And I think that's what Pixar is and was.

1:09:46.5 SC: Well, the idea of the digital convergence, the idea that things have gone from being analog to being digital, and that opens up new ways of dealing with them and manipulating them and so forth. Some people are going to say, "Okay, but the digital representation of the information is never perfect. It's only just an approximation or something like that".

1:10:06.6 AS: No, I don't believe that. I know there are true believers who think vinyl cannot be, you know, nothing can be as good as vinyl when it comes to audio. I just think that's wrong. The theory says if you do digital representation correctly, and nobody does, by the way, but if they did, they could simulate vinyl. But it's probably cheaper just to buy vinyl if you want that.

1:10:33.7 SC: Probably.

1:10:35.3 AS: And the same goes with, well, with any recording medium, you can just say the original film is just better because it's got this graininess to it. Okay, if that becomes your definition of good, we can add that noise back in if you really want, but I don't think anybody's going to bother, right?

1:11:01.9 SC: Well, I think this is, it's late in the podcast so we can, you know, indulge ourselves a little bit. And I really want the audience to understand your conception of what a pixel is, because this is, it's interesting because you clearly think about pixels differently than I do. I think I used to think about them the same way everyone else does, as little squares on a grid and you're teaching us.

1:11:25.3 AS: Pixels are not little squares. They never have been, ever.

1:11:27.2 SC: Not little squares of the grid?

1:11:29.2 AS: They've never been little squares.

[laughter]

1:11:31.7 SC: This is a deep concept.

1:11:32.6 AS: Everybody suffers from that. I mean, you can't see a pixel just to start with. That kind of sets people back. What am I looking at right now? And I'm looking at Sean on my screen. Well, you're looking at display elements. Well, display elements are these little glowy pieces of light that aren't, they have a shape. They're continuous analog blobs of light. They're controlled digitally on a grid. They're driven by a pixel. But they're just little blobs of light. And those little blobs of light intersect one another to form a, the picture I'm looking at is not broken up into little anythings. It's a continuous surface. That's what we mean by a picture. Now, people say, "Yeah, but if you zoom in, you'll see the pixels." Well, that's a dirty trick. It never has been true. If you zoom in, every app has a zoom feature. And I think this is one of the reasons people think that they're looking at little squares is 'cause if you zoom in using a zoom feature on most apps, you eventually get to the point where you're looking at an array of little squares. Oh, it must be little squares down there because I zoomed and there they are.

1:12:54.0 AS: But no, that's a dirty trick. It's a real simple hardware trick to replace each. Suppose you wanted to zoom by a factor of 100. Just replace each pixel with a 100 by 100 square array. And that gives you a picture that's 100 times bigger in both dimensions. But what you've done is replace a little pixel with a 100 by 100 square array of pixels, which guess what? It looks like a little square. What a surprise. But it's not a picture of a pixel up close. It's a picture of, now what is that, 10,000 pixels.

1:13:32.7 SC: So what is a pixel?

1:13:34.8 AS: It's a sample. So a sample is, so I like to think of a picture like when I'm looking at you right now is a crumpled sheet. So I'm going to, you usually look at it from the side, but if you turn the sheet sideways and look down on it, the heights of the crumples in the crumpled sheet are the brightnesses of the picture at the various points. At an infinity of points, because there are an infinity of points in this crumpled sheet. Now, you take a bed of nails where the nails are equally spaced in both dimensions, and you let this crumpled sheet settle onto this bed of nails. And then you throw away everything on the sheet except the points that land on the nails. Those are the pixels, the samples of this continuous surface on a regularly spaced grid. It's just a number taken from the continuum.

1:14:45.8 SC: So it's an abstract informational concept. It's not a physical concept?

1:14:48.7 AS: It is. You can't see it. It's just a...

1:14:51.6 SC: That's so you know you can't see it.

1:14:51.7 AS: Number stored in a piece of memory, right? All right. You can't see it. So like I like to say, I hold up my phone, I say, I've got billions of pixels in this phone, but I can't see any of them. You can't either. On your phone, you probably have billions of pixels too. You can't see them. They're there. They're in a file. Because that's how pixels are stored. They're stored as bits in a file. To see a picture, you click on the file name or whatever, and suddenly there it is on your laptop or your cell phone. What's just happened? Well, a very fast computation has just happened where that array of pixels has been reconstructed using the sampling theorem into the image that it originally represented. And the sampling theorem is a mind blower, but it basically says if you do that crumpled sheet on a bed of nails trick the right way, you can throw away all the information. It seems like you're throwing away an infinity of information between the nails and not losing it. And not losing it is the mind-blowing mathematical trickery. You don't lose it. Those little blobs of light happen to be the almost exact theoretical shape needed to reconstruct that the theorem requires. And at that point, you have to go off and do the details of the theorem.

1:16:24.5 SC: I don't wanna go into all the details of the theorem, but maybe a detail or two would do us good here, because it fascinated me in the book discussion because you actually had to care about the difference between countable infinity and uncountable infinity.

1:16:40.1 AS: Oh, you want me to do that? All right. [laughter]

1:16:41.8 SC: I do. [laughter]

1:16:43.2 AS: Okay. So the sampling theorem, by the way, there was a fight over who invented the sampling theorem, and I came to a solution in my book. It was a very surprising solution.

1:16:54.5 SC: I was convinced by your story. It was a Russian.

1:16:56.6 AS: Yeah, it was a Russian commie. Yeah. Okay.

[chuckle]

1:17:01.8 AS: He was a really smart guy. He said, you can sample, as I explained the sheet on the sharp nails, and then later you can, if you put a certain shape on each of those nails and add up the shapes that you put on it, you get the original picture back. And the shape, he said, that you need to use looks like this. Well, it's a peak with little ripples out to each side. Like if you drop a rock into a... Pebble into still water, it'll go out in circles of diminishing heights.

1:17:42.1 SC: So it's kind of like a bell curve, except the tails of it get wavy rather than just the...

1:17:45.7 AS: Yeah, they're wiggly. Okay. And they're small compared to the major hump in the middle, but they're there. Well, in the real world, we can't have things that go out to infinity in both directions. We have to have things of finite size. So we, you know, the engineers of the world have said, "Well, you can replace that hump with the infinity of wiggles with a hump with one or two wiggles and pretty much have the same effect as the full-blown shape," and that's what...

1:18:26.0 SC: The central idea seems to be that this, on the bed of nails, you're measuring the height of the paper at each of your little points, but you don't reconstruct it by constructing like a little, you know, piecewise constant thing where you just add up...

1:18:43.9 AS: You certainly don't put a little square there. That's it. You put...

1:18:45.7 SC: You don't put a little square there. You put this shape there and you add up all the shapes. And as long as you don't have too many high-frequency wiggles, you can completely convey the smooth image in a discrete amount of information.

1:19:00.4 AS: That's true. And getting... You know, you put in the caveat, you've got to, in order to honor the theorem, you have to be sure to get rid of the super high frequencies 'cause they'll mess everything up. But people know how to do that.

1:19:16.4 SC: Yeah. So, I mean, those are super short distances that the eye cannot perceive anyway. So if we lose that, it's not a big deal.

1:19:22.3 AS: That's the idea, right? That you put it off into parts, into noise that don't make humans uncomfortable.

1:19:29.3 SC: And this is all... I mean, I love it because it's all part of the technical side that most people don't need to know about, but you can capture.

1:19:38.7 AS: It's beautiful. If you can follow the theorem, it's so gorgeous in that mathematical way. This theorem, which is just completely non-intuitive, works. And we use it. We're using it right now. Everybody uses it all the time. It's what made the modern world work.

1:19:57.8 SC: Well, and you also help me understand why certain digitizations of old movies are not very good because they didn't use the sampling theorem correctly. [chuckle]

1:20:07.2 AS: They didn't. They didn't know what they were doing. They really did use a little square model. One thing that annoys me, I watch a lot of TV, and the one thing that continues to annoy me is smooth motions jerk. Allison, my wife, doesn't see it. I'm sitting there going, "Damn it. There's a jerk." She says...

1:20:27.5 SC: [laughter] You're a professional, right?

1:20:29.7 AS: She didn't see it, right? I said, "Well, I can't believe you don't see it." To me, they're just awful, these jerks instead of a smooth motion. But it means to me, like, the person who's converting from old film format to modern video is not doing the frame conversions correctly according to the sampling theorem. So they're putting in these, oh, they're putting in approximations that don't work if somebody sensitive like me is watching. That's what I'm saying.

1:21:00.7 SC: Well, but I think it's kind of reassuring to the person on the street to know that this kind of technical detail helps the conversion from analog to digital be much more faithful than we might have thought.

1:21:15.1 AS: Yeah, it can be. If you're willing to...

1:21:17.4 SC: If you do it right.

1:21:18.8 AS: If you do it right, it can be extremely accurate, extremely accurate. Another place where I'm sort of surprised that people don't object to the lack of appropriate reconstruction, sampling and reconstruction is spinning wheels on cars. Do you see them go backwards?

1:21:40.1 SC: Oh, yeah. Sometimes, yeah.

1:21:41.2 AS: They go backwards all the time.

[laughter]

1:21:43.4 AS: And I'm going, "No, they don't. Doesn't that bother you?" That's an artifact.

1:21:50.8 SC: Well, does someone like you need to know a lot about how biological vision works or the limitations from the eye and from the visual cortex?

1:21:58.6 AS: I thought so. That's why I spent a lot of years learning how the color system works, color theory, but it turned out not to be all that useful. The closest I got to, you know, I made a contribution, which it's the HSV to RGB color transform. When I started working at Xerox PARC with my friend Dick Schaap, his machine had red, green, and blue primaries because that's how video works, red, green, and blue. You adjust the voltages on the three guns and you get all the colors possible with those three primaries. And I went to him and I said, "Dick, do you know how to convert RGB to hue, saturation, and value?" Because we artists don't think in terms of RGB. We choose a hue from the color circle and we whiten it with white or we darken it with black. And he said, "Well, nobody's ever done that. Why don't you do that?"

[laughter]

1:23:01.4 AS: So I did.

1:23:02.3 SC: Someone had to be first.

1:23:02.5 AS: It was my first contribution. I went home and did it overnight. It was real easy. What amazes me is it's still in use. Now, hue, saturation, value sounds like how the human perceptual system works.

1:23:15.3 AS: But what I did was just a quick and dirty trick. It's not at all built on what the human eye actually does.

1:23:20.6 SC: Right. Okay.

1:23:22.5 AS: What the human eye does is awesome.

1:23:23.8 SC: Yeah. I guess, you can have a closing thought of your own. My closing thought is I certainly hope that the future of the Great Digital Convergence continues to honor both the technical side and the artistic creative side of things, as much as you were pushing it to do for this past few decades.

1:23:42.2 AS: Yeah. I guess what I'm 80, just shy of 82 as we sit here, and I've watched, I was born before computers and I've watched the whole damn thing unwind. And it's been awesome, hasn't it?

1:23:58.5 SC: [chuckle] Yeah.

1:23:58.9 AS: What really angers me is I don't have another 80 years 'cause it's gonna be equally awesome if not, you know, what's it gonna be? I want to see. And I don't get to.

1:24:12.8 SC: But there's a lot of prospect out there. And I think that you've set us up very well.

1:24:14.8 AS: Oh, yeah. I think I'm so excited. I'm excited by the art possibilities. One of the first things I did when I started hearing about the mid-journey and the picture versions of AI, I called up one of my buddies, David M, who's been making computer art as long as I can remember. He came and did computer, some of the earliest computer art on the paint program at Xerox PARC. And I've known him ever since. And I know some of his early computer renderings have now been used by Herbie Hancock for his album covers.

1:25:04.8 SC: Oh, nice.

1:25:06.2 AS: So I call and say, "What do you think of this AI stuff?" He says, "Alvy, it's fantastic." I says, "You're not threatened by that?" He goes, "Absolutely not." He says, "You know, it's just wrong to think that what artists are are renderers, draftsmen. No, we're not." We're basically, we're the critics for the human race, right? That's not his terminology. That's mine. But we can say, this is boring. This is trite. You know, and most of the stuff that these programs turn out is boring and trite. But ever so often, and it's, you know, he uses his skill and taste and education. He says, "But look at this. This is fantastic." And he'll urge the machine to go off on that direction and explore it some more. Again, it's being used as a lever.

1:25:57.7 SC: It's a tool. Yeah.

1:25:58.8 AS: Yeah, which if people like that start saying, "My job is threatened as a creator," then I'll get worried about it. But I haven't heard any of that yet.

1:26:10.7 SC: Yeah. All right. That's good to know. Alvy Ray Smith, thanks so much for being on the Mindscape podcast. It was a lot of fun.

1:26:15.6 AS: Bye-bye.

[music]