#62 - Häggström on AI Motivations and Risk Denialism

In this episode I talk to Olle Häggström. Olle is a professor of mathematical statistics at Chalmers University of Technology and a member of the Royal Swedish Academy of Sciences (KVA) and of the Royal Swedish Academy of Engineering Sciences (IVA). Olle’s main research is in probability theory and statistical mechanics, but in recent years he has broadened his research interests to focus applied statistics, philosophy, climate science, artificial intelligence and social consequences of future technologies. He is the author of Here be Dragons: Science, Technology and the Future of Humanity (OUP 2016). We talk about AI motivations, specifically the Omohundro-Bostrom theory of AI motivation and its weaknesses. We also discuss AI risk denialism. You can download the episode here or listen below. You can also subscribe to the podcast on Apple Podcasts, Stitcher and a variety of other podcasting services (the RSS feed is here). Show Notes 0:00 - Introduction 2:02 - Do we need to define AI? 4:15 - The Omohundro-Bostrom theory of AI motivation 7:46 - Key concepts in the Omohundro-Bostrom Theory: Final Goals vs Instrumental Goals 10:50 - The Orthogonality Thesis 14:47 - The Instrumental Convergence Thesis 20:16 - Resource Acquisition as an Instrumental Goal 22:02 - The importance of goal-content integrity 25:42 - Deception as an Instrumental Goal 29:17 - How the doomsaying argument works 31:46 - Critiquing the theory: the problem of self-referential final goals 36:20 - The problem of incoherent goals 42:44 - Does the truth of moral realism undermine the orthogonality thesis? 50:50 - Problems with the distinction between instrumental goals and final goals 57:52 - Why do some people deny the problem of AI risk? 1:04:10 - Strong versus Weak AI Scepticism 1:09:00 - Is it difficult to be taken seriously on this topic?   Relevant Links Olle's Blog  Olle's webpage at Chalmers University 'Challenges to the Omohundro-Bostrom framework for AI Motivations' by Olle (highly recommended) 'The Superintelligent Will' by Nick Bostrom 'The Basic AI Drives' by Stephen Omohundro Olle Häggström: Science, Technology, and the Future of Humanity (video) Olle Häggström and Thore Husveldt debate AI Risk (video) Summary of Bostrom's theory (by me) 'Why AI doomsayers are like sceptical theists and why it matters' by me   #mc_embed_signup{background:#fff; clear:left; font:14px Helvetica,Arial,sans-serif; } /* Add your own MailChimp form style overrides in your site stylesheet or in this style block. We recommend moving this block and the preceding CSS link to the HEAD of your HTML file. */ Subscribe to the newsletter