SVEN NYHOLM is Professor of the Ethics of Artificial Intelligence at the Ludwig Maximilian University of Munich, a member of the Ethics Advisory Board of the Human Brain Project, and an Associate Editor of Science and Engineering Ethics. He is the author of Humans and Robots: Ethics, Agency, and Anthropomorphism and Revisiting Kant's Universal Law and Humanity Formulas.

Preface xiii

Acknowledgments xvii

1 What is Technology (From an Ethical Point of View)? 1

1.1 A Hut in the Black Forest 1

1.2 The Question Concerning Technology: The Instrumental Theory of Technology from Martin Heidegger to Joanna Bryson 3

1.3 "Post- Phenomenology" and the Mediation Theory of Technology 7

1.4 Technologies Conceived of as Being More Than Mere Means or Instruments 11

1.5 Technologies Regarded as Moral Agents 13

1.6 Technologies Regarded as Moral Patients 16

1.7 Some of the Key Types of Technologies That Will Be Discussed at Greater Length in Later Chapters of the Book 19

Annotated Bibliography 24

2 What is Ethics? (and, in Particular, What is Technology Ethics)? 25

2.1 Two Campaigns 25

2.2 The Ethics of Virtue and Human Flourishing in Ancient Greece 28

2.3 Ancient Chinese Confucianism and Traditional Southern African Ubuntu Ethics 32

2.4 Kantian Ethics 36

2.5 Utilitarianism and Consequentialist Ethical Theories 39

2.6 If Ethics More Generally Can Be All the Things Discussed in the Previous Sections, then What Does this Mean for Technology Ethics in Particular? 44

2.7 How Technology Ethics Can Challenge and Create a Need for Extensions of More General Ethical Theory 46

Annotated Bibliography 49

3 Methods of Technology Ethics: The Ethics of Self- Driving Cars as a Case Study 51

3.1 Methodologies of Ethics? 51

3.2 The Ethics of Self- Driving Cars 53

3.3 Ethics by Committee 56

3.4 Ethics by Analogy: The Trolley Problem Comparison 58

3.5 Empirical Ethics 61

3.6 Applying Traditional Ethical Theories 65

3.7 Which Method(s) Should We Use in Technology Ethics? Only One or Many? 70

Annotated Bibliography 74

4 Artificial Intelligence, Value Alignment, and the Control Problem 76

4.1 Averting a Nuclear War 76

4.2 What Is Artificial Intelligence and What Is the Value Alignment Problem? 79

4.3 The Good and the Bad, and Instrumental and Non- Instrumental Values and Principles 83

4.4 Instrumentally Positive Value- Alignment of Technologies 86

4.5 Instrumentally Negative Misalignment of Technologies 87

4.6 Positive Non- Instrumental Value Alignment of Technologies 90

4.7 Negative Non- Instrumental Value Misalignment of Technologies 94

4.8 The Control Problem 96

4.9 Control as a Value: Instrumental or Non- Instrumental? And Are There Some Technologies It Might Be Wrong to Try to Control? 99

Annotated Bibliography 102

5 Behavior Change Technologies, Gamification, Personal Autonomy, and the Value of Control 104

5.1 A Better You? 104

5.2 Behavior Change Technologies and Gamification 107

5.3 Control: Three Basic Observations 110

5.4 Key Dimensions of Control Discussed in Different Areas of Philosophy 112

5.5 Behavior Change Technologies and the "Subjects" and "Objects" of Control 116

5.6 The Value and Ethical Importance of Control 120

5.7 Concluding This Chapter 123

Annotated Bibliography 125

6 Responsibility and Technology: Mind the Gap(s)? 127

6.1 Two Events 127

6.2 What Is Responsibility? Different Ways in Which People Can Be Held Responsible and Different Things for Which People Can Be Held Responsible 130

6.3 Responsibility Gaps: General Background 134

6.4 Responsibility Gaps Created by Technologies 138

6.5 Filling Responsibility Gaps by Having People Voluntarily Take Responsibility 142

6.6 Should We Perhaps Welcome Responsibility Gaps? 145

6.7 Responsible Machines? 148

6.8 Human-Machine Teams and Responsibility 153

6.9 Concluding This Chapter 155

Annotated Bibliography 156

7 Can a Machine be a Moral Agent? Should any Machines be Moral Agents? 158

7.1 Machine Ethics 158

7.2 Arguments in Favor of Machine Ethics and Types of Artificial Moral Agents 160

7.3 Objections to the Machine Ethics Project 163

7.3.1 First Objection: Morality Cannot Be Fully Codified 164

7.3.2 Second Objection: It Is Unethical to Create Machines that We Allow to Make Life- and- Death Decisions About Human Beings 165

7.3.3 Third Objection: Moral Agents Need to Have Moral Emotions and Machines Do Not/Cannot Have Emotions 167

7.3.4 Fourth Objection: Machines Are Not Able to Act for Reasons 169

7.3.5 Brief Reminder of the Objections to Machine Ethics Considered Above 171

7.4 Possible Ways of Responding to the Critiques of the Machine Ethics Project 172

7.4.1 First Response: Bottom- Up Learning Rather Than Top- Down Rule- Following 173

7.4.2 Second Response: Resisting the Idea That Machines/Technologies Should Ever Be Full Moral Agents 175

7.4.3 Third Response: Switching to Thinking in Terms of Human-Machine Teams Rather Than in Terms of Independent Artificial Moral Agents 176

7.5 Concluding This Chapter 179

Annotated Bibliography 180

8 Can Robots be Moral Patients, with Moral Status? 182

8.1 The Tesla Bot and Erica the Robot 182

8.2 What Is a Humanoid Robot? And Why Would Anybody Want to Create a Humanoid Robot? 186

8.3 Can People Act Rightly or Wrongly Toward Robots? 190

8.4 Can Robots Have Morally Relevant Properties or Abilities? 193

8.5 Can Robots Imitate or Simulate Morally Relevant Properties or Abilities? 197

8.6 Can Robots Represent or Symbolize Morally Relevant Properties or Abilities? 200

8.7 Should We Be Discussing- Or Perhaps Better Be Avoiding- the Question of Whether Robots Can Be Moral Patients, with Moral Status? 205

Annotated Bibliography 210

9 Technological Friends, Lovers, and Colleagues 213

9.1 Replikas, Chuck and Harmony, and Boomer 213

9.2 Ethical Issues That Arise in This Context Independently of Whether Technologies Can Be Our Friends, Lovers, or Colleagues 216

9.3 Technological Friends 219

9.4 Technological Lovers and Romantic Partners 222

9.5 Robotic Colleagues 227

9.6 Are These All- or- Nothing Matters? Respect for Different Points of View 232

9.7 The Technological Future of Relationships 234

Annotated Bibliography 238

10 Merging with the Machine: The Future of Human-Technology Relations 240

10.1 The Experience Machine 240

10.2 Different Ways of Merging with- Or Merging with the Help of- Technology 244

10.3 Transhumanism, Posthumanism, and Whether We Should Become- Or Perhaps Already Are- Cyborgs 248

10.4 Some Critical Reflections on the Proposals to Merge with Technologies and the Arguments and Outlooks Used in Favor of Such Proposals 251

10.5 Concluding Reflections: Revisiting the Hut in the Black Forest 256

Annotated Bibliography 263

Index 265