Episode 30: Derek Leben on Ethics for Robots and Artificial Intelligences

January 21, 2019 | , ,

It’s hardly news that computers are exerting ever more influence over our lives. And we’re beginning to see the first glimmers of some kind of artificial intelligence: computer programs have become much better than humans at well-defined jobs like playing chess and Go, and are increasingly called upon for messier tasks, like driving cars. Once we leave the highly constrained sphere of artificial games and enter the real world of human actions, our artificial intelligences are going to have to make choices about the best course of action in unclear circumstances: they will have to learn to be ethical. I talk to Derek Leben about what this might mean and what kind of ethics our computers should be taught. It’s a wide-ranging discussion involving computer science, philosophy, economics, and game theory.

Support Mindscape on Patreon or Paypal.

Derek Leben received his Ph.D. in philosopy from Johns Hopkins University in 2012. He is currently an Associate Professor of Philosophy at the University of Pittsburgh at Johnstown. He is the author of Ethics for Robots: How to Design a Moral Algorithm.

14 thoughts on “Episode 30: Derek Leben on Ethics for Robots and Artificial Intelligences”

  1. 0:43:11 SC: If I understand it correctly, the Nash equilibrium is one where one person cannot unilaterally change to get a better outcome without hurting somebody else, but a Pareto improvement would be where, if we all change at once, we will all be better off, is that right?

    0:43:29 DL: That’s exactly right, […]

    This is not “exactly right”. It has nothing to do with hurting other parties, that’s irrelevant. You are in a Nash Equilibrium when any change to your strategy you can enact will be counter to your interests (payoff function). In the prisoner dilemma game you defect because it doesn’t matter what your adversary does, you still benefit – either he defects too in which case you avoid a long sentence or he stays quiet in which case you get the lowest sentence. To do otherwise it to guarantee you cannot get the lowest sentence and open yourself to the possibility that you may get the highest.

  2. I see the value in thinking deeply about programing machines so that there are ethical guidelines to prevent unnecessary harm. However, when I am driving my car, if I recognize that a collision is about to occur, my instinct will be self preservation. I think that car manufacturers will recognize this human tendency and will present these to the public as designed to maximize personal safety. It is doubtful that buyers will trade their own autonomy for a machine that might have a different priority.

  3. How many times a year does a human driver have to think if she should run into a bicyclist wearing a helmet or her friend who isn’t in order to swerve to avoid a 90 year man who just walked in front of her?

    99.999% of accidents have nothing to do with the trolley car dilemma yet this was about 1/3 of the podcast.

  4. @alex

    > In the prisoner dilemma game you defect because it doesn’t matter what your adversary does, you still benefit

    > either he defects too in which case you avoid a long sentence or he stays quiet in which case you get the lowest sentence.

    That is inaccurate. In most versions of the prisoner’s dilemma (when repeated over many iterations), the Nash equilibrium is both players defecting, which leaves both players worse off than both cooperating. Any strategy other than defecting leaves the other side open to exploiting your strategy and becoming much better off. The dilemma describes a non-pareto optimal Nash equilibrium.

  5. Machines deciding where to extract natural resources to sell them also effects people in a similar ethical dilemma, there are many other illustrative examples of the problem that are not self driving cars. That is just a popular topic at this time as autonomous vehicles are beginning to affect our lives.

  6. Lol… the trolley square problem is an analogy for many problems AIs will face–it’s not specifically about running over people, or dogs, or flowers, or whatever… beep boop

  7. But people aren’t constantly running into the trolley problem when they drive so why would a driverless car? In the extremely rare instance when this might happen, it isn’t as if the human driver usually has time to think of the action to take between two bad outcomes so why would we expect the driverless car to do better?

    I liked the rest of the podcast, though.

  8. In the discussion of various philosophical theories for the programing of robots, it is important to realize that they do not understand anything. It is not possible to program them to understand. Therefore, we need to restrict their functions to areas where understanding is not required. Perhaps understanding is not required in the driving of automobiles or semitrailer trucks on crowded roads and neighborhood streets, but we need to make sure.

  9. atheist4thecause

    I’m not a big fan of the absolutism of what actions are and aren’t harm. If one was trying to create an algorithm of harm, they would have to consider the impact of the action as well the impact on society. Stealing $50 from the President, for instance, isn’t as likely to harm the President as a uneducated poor person trying to take a train to work. At the same time, the President is much more valuable to society than the uneducated poor person.

    Now, we’d have to find a way to mesh what I call the Moral Value and Ethical Value. In my above example, despite the President being probably millions of times more beneficial to society, the harm of losing $50 would be basically 0, and so the uneducated poor worker, who would be harmed greatly by not being able to get to work, would end up being harmed more in the Combined Value.

    Another issue about the absolutism about what is being talked about is with the Trolley Problem, when we switch the track over we don’t actually know that is going to kill someone. Heck, we don’t even know pushing someone in front of a trolley is going to kill them. Our actions all lead to a probability someone is harmed or killed. You would likely have to take the probability of harm, weigh the extensiveness of it, and mesh that into the Combined Value as well.

  10. Sean,
    You state at [1:09:11] that you do not fully buy into utilitarian arguments, in part, due to the “Utility Monster” and “Repugnant Conclusion” criticisms. I believe both of these responses have been addressed within utilitarian circles.

    Utility Monster:
    I argue that this criticism is not a bug, but a feature of utilitarianism working correctly.

    In the “Utility Monster” thought experiment, we are asked to:
    1) Imagine an individual capable of generating greater utility than thousands of other individuals combined
    2) Observe that utilitarianism demands we devote significant resources to pleasing that one, even if this comes at great expense to the many

    We are left to conclude that 2 demonstrates a flaw in utilitarianism.

    The true issue is that limits in human imaginations result in many people struggling to accomplish 1. We often fail to envision a human who could act as a utility monster in relation to 10,000 other humans. We are left wondering “What makes them so special?” This is a flaw that is smuggled into the framing of the thought experiment.

    In situations where the utility monster truly is accepted as being capable of generating greater utility than the many, the results are not so unintuitive.
    -A single human is a utility monster in relation to 10,000 ants.
    -A single ant is a utility monster in relation to 10,000 bacteria.
    -A single bacterium is a utility monster in relation to 10,000 grains of sand.

    These conclusions are not nearly so controversial.

  11. Repugnant Conclusion:
    Parfit’s Repugnant Conclusion presents a very strong argument against attempting to calculate utility through “mere addition.” The implication is that a massive group of miserable people is greater than a small group of people who are very content.

    For this reason, I am more supportive of calculating utility using a mean (arithmetic, or perhaps geometric). *As a side note, I personally appreciate that this approach does not imply that adding more people is always desirable (as advocated by former guest, Tyler Cowen).

    Upon applying his reasoning to this model, Parfit reasonably points out that a very small number of extremely high utility people (possibly just one person) would be preferable to billions of people who produce even slightly less utility on average. Again, the discomfort generated by this conclusion is due to lapses in imagination. We are unable to easily imagine a world where a single individual could reliable generate higher average utility than a large society. Even if we could, there would be meaningful moral implications for any plan to transition from our current situation, containing billions of moral agents down to very few.

    A frequent criticism of averaging utility functions is the claim that “average utility could be increased by killing everyone who is generating low utility.” There is a very reasonable response to this criticism.
    A moral community includes all people ever to exist, be they in the past, present, or future. Even if committing genocide could lead to greater average utility amongst those who remained, the utility of those who were killed must still be factored in. One would be unlikely to achieve a net gain in average utility after accounted for the negative utility generated during the mass slaughter.

  12. I’m interested in alternative utility aggregation functions beyond utilitarian (sum of utility) and maximin. It seems like there should be a large number of such functions to consider, some which may have better tradeoffs – better alignment with intuitive preferable choices. One which I like is sum(sqrt(u)). This seems to strike a good balance to me, preferring to increase the utility of the lowest, but also providing some (reduced) guidance for all population members.

  13. My feeling is that pragmatically these AIs will seek to optimize legality and minimize liability – with no consideration of ethics. In some ways, that’s a less interesting discussion and just defers the problem to the legislative process. It also opens questions about whose liability should be optimized? The system operator/owner? manufacturer?

Comments are closed.

Sean Carroll hosts conversations with the world's most interesting thinkers. Science, society, philosophy, culture, arts, and ideas.

Listen/Subscribe via:

Archives by Month

Archives by Category

Search Podcast Archives