Existential risk from transformative AI: an economic perspective
Abstract
The prospective arrival of transformative artificial intelligence (TAI) will be a filter for the human civilization – a threshold beyond which it will either strongly accelerate its growth, or vanish. Historical evidence on technological progress in AI capabilities and economic incentives to pursue it suggest that TAI will most likely be developed in just one to four decades. In contrast, theoretical problems of AI alignment, needed to be solved in order for TAI to be “friendly” towards humans rather than cause our extinction, appear difficult and impossible to solve by mechanically increasing the amount of compute. This means that transformative AI poses an imminent existential risk to the humankind which ought to be urgently addressed. Starting from this premise, this paper provides new economic perspectives on discussions surrounding the issue: whether addressing existential risks is cost effective and fair towards the contemporary poor, whether it constitutes “Pascal’s mugging”, how to quantify risks that have never materialized in the past, how discounting affects our assessment of existential risk, and how to include the prospects of upcoming singularity in economic forecasts. The paper also suggests possible policy actions, such as ramping up public funding on research on existential risks and AI safety, and improving regulation of the AI sector, preferably within a global policy framework.
First published online 10 July 2024
Keyword : transformative artificial intelligence, artificial general intelligence, alignment, existential risk, long-run economic growth, longtermism
How to Cite
Growiec, J. (2024). Existential risk from transformative AI: an economic perspective. Technological and Economic Development of Economy, 30(6), 1682–1708. https://doi.org/10.3846/tede.2024.21525
This work is licensed under a Creative Commons Attribution 4.0 International License.
References
Acemoglu, D., & Autor, D. (2011). Skills, tasks and technologies: Implications for employment and earnings. In O. Ashenfelter & D. Card (Eds.), Handbook of labor economics (vol. 4, pp. 1043–1171). Elsevier. https://doi.org/10.1016/S0169-7218(11)02410-5
Acemoglu, D., & Restrepo, P. (2018). The race between man and machine: Implications of technology for growth, factor shares and employment. American Economic Review, 108(6), 1488–1542. https://doi.org/10.1257/aer.20160696
Albanesi, S., Dias da Silva, A., Jimeno, J. F., Lamo, A., & Wabitsch, A. (2023). New technologies and jobs in Europe. (Working Paper No. 31357). National Bureau of Economic Research. https://doi.org/10.3386/w31357
Aschenbrenner, L. (2020). Existential risk and growth (Working Paper No. 6). Columbia University and Global Priorities Institute, University of Oxford. https://globalprioritiesinstitute.org/wp-content/uploads/Leopold-Aschenbrenner_Existential-risk-and-growth_.pdf
Autor, D., Dorn, D., Katz, L., Patterson, C., & Van Reenen, J. (2020). The fall of the labor share and the rise of superstar firms. The Quarterly Journal of Economics, 135(2), 645–709. https://doi.org/10.1093/qje/qjaa004
Barro, R. J. (2003). Determinants of economic growth in a panel of countries. Annals of Economics and Finance, 4, 231–274. https://down.aefweb.net/WorkingPapers/w505.pdf
Bloom, N., Jones, C. I., Van Reenen, J. & Webb, M. (2020). Are ideas getting harder to find? American Economic Review, 110(4), 1104–1144. https://doi.org/10.1257/aer.20180338
Bostrom, N. (2014). Superintelligence: Paths, dangers, strategies. Oxford University Press.
Bostrom, N., Douglas, T., & Sandberg, A. (2016). The unilateralist’s curse and the case for a principle of conformity. Social Epistemology, 30(4), 350–371. https://doi.org/10.1080/02691728.2015.1108373
Branwen, G. (2022, January 2). The scaling hypothesis. https://gwern.net/scaling-hypothesis
Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S. M., Nori, H., Palangi, H., Ribeiro, M. T. & Zhang, Y. (2023). Sparks of artificial general intelligence: Early experiments with GPT-4. ArXiv:2303.12712. https://doi.org/10.48550/arXiv.2303.12712
Chichilnisky, G. (2000). An axiomatic approach to choice under uncertainty with catastrophic risks. Resource and Energy Economics, 22(3), 221–231. https://doi.org/10.1016/S0928-7655(00)00032-4
Chichilnisky, G., Hammond, P. J., & Stern, N. (2020). Fundamental utilitarianism and intergenerational equity with extinction discounting. Social Choice and Welfare, 54, 397–427. https://doi.org/10.1007/s00355-019-01236-z
Cotra, A. (2020). Draft report on AI timelines. AI Alignment Forum. https://www.alignmentforum.org/posts/KrJfoZzpSDpnrv9va/draft-report-on-ai-timelines
Cotra, A. (2022). Two-year update on my personal AI timelines. AI Alignment Forum. https://www.alignmentforum.org/posts/AfH2oPHCApdKicM4m/two-year-update-on-my-personal-ai-timelines
Davidson, T. (2021). Could advanced AI drive explosive economic growth? Open Philanthropy. https://www.openphilanthropy.org/research/could-advanced-ai-drive-explosive-economic-growth/
Davidson, T. (2023) What a compute-centric framework says about takeoff speeds. Open Philanthropy. https://www.openphilanthropy.org/research/what-a-compute-centric-framework-says-about-takeoff-speeds/
Eloundou, T., Manning, S., Mishkin, P., & Rock, D. (2023). GPTs are GPTs: an early look at the labor market impact potential of large language models (Working Paper No. 2303.10130). Arxiv.org. https://doi.org/10.48550/arXiv.2303.10130
Etzioni, O. (2016). No, the experts don’t think superintelligent AI is a threat to humanity. MIT Technology Review. https://www.technologyreview.com/2016/09/20/70131/no-the-experts-dont-think-superintelligent-ai-is-a-threat-to-humanity/
Frey, C. B., & Osborne, M. (2017). The future of employment: How susceptible are jobs to computerisation? Technological Forecasting and Social Change, 114, 254–280. https://doi.org/10.1016/j.techfore.2016.08.019
Gordon, R. J. (2016). The rise and fall of American growth: The U.S. standard of living since the Civil War. Princeton University Press. https://doi.org/10.1515/9781400873302
Grace, K. (2022). Let’s think about slowing down AI. Less Wrong. https://www.lesswrong.com/posts/uFNgRumrDTpBfQGrs/let-s-think-about-slowing-down-ai
Grace, K., Stewart, H., Sandkühler, J. F., Thomas, S., Weinstein-Raun, B., & Brauner, J. (2024). Thousands of AI authors on the future of AI. Arxiv:2401.02843. https://doi.org/10.48550/arXiv.2401.02843
Growiec, J. (2022a). Accelerating economic growth: lessons from 200 000 years of technological progress and human development. Springer. https://doi.org/10.1007/978-3-031-07195-9
Growiec, J. (2022b). Automation, partial and full. Macroeconomic Dynamics, 26(7), 1731–1755. https://doi.org/10.1017/S1365100521000031
Growiec, J. (2023). What will drive global economic growth in the digital age? Studies in Nonlinear Dynamics and Econometrics, 27(3), 335–354. https://doi.org/10.1515/snde-2021-0079
Gruetzemacher, R. & Whittlestone, J. (2021). The transformative potential of artificial intelligence. Arxiv:1912.00747. https://doi.org/10.48550/arXiv.1912.00747
Hanson, R., & Yudkowsky, E. (2013). The Hanson-Yudkowsky AI-foom debate. Machine Intelligence Research Institute. https://intelligence.org/files/AIFoomDebate.pdf
Harari, Y. N. (2014). Sapiens: A brief history of humankind. Vintage.
Hawking, S., Russell, S., Tegmark, M., & Wilczek, F. (2014, May 1). Stephen Hawking: ‘Transcendence looks at the implications of artificial intelligence - but are we taking AI seriously enough?’. Independent. https://www.independent.co.uk/news/science/stephen-hawking-transcendence-looks-at-the-implications-of-artificial-intelligence-but-are-we-taking-ai-seriously-enough-9313474.html
Hendrycks, D. (2023). Natural selection favors AIs over humans. Arxiv: 2303.16200v4. https://doi.org/10.48550/arXiv.2303.16200
Hilbert, M., & López, P. (2011). The world’s technological capacity to store, communicate, and compute information. Science, 332(6205), 60–65. https://doi.org/10.1126/science.1200970
Hilton, B. (2022). Preventing an AI-related catastrophe. 80 000 Hours. https://80000hours.org/problem-profiles/artificial-intelligence/
Johansen, A. & Sornette, D. (2001). Finite-time singularity in the dynamics of the world population, economic and financial indices. Physica A: Statistical Mechanics and its Applications, 294(3–4), 465–502. https://doi.org/10.1016/S0378-4371(01)00105-4
Jones, C. I. (2023). The AI dilemma: Growth versus existential risk (Working Paper No. 31837). National Bureau of Economic Research. https://doi.org/10.3386/w31837
Klump, R., McAdam, P., & Willman, A. (2012). The normalized CES production function: Theory and empirics. Journal of Economic Surveys, 26(5), 769–799. https://doi.org/10.1111/j.1467-6419.2012.00730.x
Korinek, A. (2023). Language models and cognitive automation for economic research. (Working Paper No. 30957). National Bureau of Economic Research. https://doi.org/10.3386/w30957
Korinek, A., & Juelfs, M. (2022). Preparing for the (non-existent?) future of work. In J. B. Bullock, Y.-C. Chen, J. Himmelreich, V. M. Hudson, A. Korinek, M. M. Young, & B. Zhang (Eds.), The Oxford handbook of AI governance. Oxford University Press. https://doi.org/10.1093/oxfordhb/9780197579329.013.44
Kosinski, M. (2023). Theory of mind may have spontaneously emerged in large language models. Arxiv: 2302.02083. https://doi.org/10.48550/arXiv.2302.02083
Krakovna, V., Uesato, J., Mikulik, V., Rahtz, M., Everitt, T., Kumar, R., Kenton, Z., Leike, J., & Legg, S. (2020, April 21). Specification gaming: The flip side of AI ingenuity. DeepMind. https://www.deepmind.com/blog/specification-gaming-the-flip-side-of-ai-ingenuity
Kurzweil, R. (2005). The singularity is near: When humans Transcend biology. Penguin.
Leike, J., & Sutskever, I. (2023, July 5). Introducing superalignment. OpenAI. https://openai.com/blog/introducing-superalignment
Martin, I., & Pindyck, R. S. (2015). Averting catastrophes: The strange economics of Scylla and Charybdis. American Economic Review, 105(10), 2947–2985. https://doi.org/10.1257/aer.20140806
McAskill, W. (2022). What we owe the future: A million-year view. Basic Books.
Milanovic, B. (2016). Global inequality: A new approach for the age of globalization. Harvard University Press. https://doi.org/10.4159/9780674969797
Muehlhauser, L., & Salamon, A. (2012). Intelligence explosion: Evidence and import. In A. Eden, J. Soraker, J. H. Moor, & E. Steinhart (Eds.), Singularity hypotheses: A scientific and philosophical assessment (pp. 15–42). Springer. https://doi.org/10.1007/978-3-642-32560-1_2
Nagy, B., Farmer, J. D., Trancik, J. E., & Gonzales, J. P. (2011). Superexponential long-term trends in information technology. Technological Forecasting and Social Change, 78(8), 1356–1364. https://doi.org/10.1016/j.techfore.2011.07.006
Nordhaus, W. D. (2021). Are we approaching an economic singularity? Information technology and the future of economic growth. American Economic Journal: Macroeconomics, 13(1), 299–332. https://doi.org/10.1257/mac.20170105
Organization for Economic Cooperation and Development. (2024). Real GDP long term forecast. OECD. https://data.oecd.org/gdp/real-gdp-long-term-forecast.htm
OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., Avila, R., Babuschkin, I., Balaji, S., Balcom, V., Baltescu, P., Bao, H., Bavarian, M., Belgum, J. …, Zoph, B. (2023). GPT-4 technical report. Arxiv: 2303.08774. https://doi.org/10.48550/arXiv.2303.08774
Ord, T. (2020). The precipice: Existential risk and the future of humanity. Hachette.
Parfit, D. (1984). Reasons and persons. Oxford University Press. https://www.stafforini.com/docs/Parfit%20-%20Reasons%20and%20persons.pdf
Parteka, A., & Kordalska, A. (2023). Artificial intelligence and productivity: Global evidence from AI patent and bibliometric data. Technovation, 125, Article 102764. https://doi.org/10.1016/j.technovation.2023.102764
Phillips, P. J., Hahn, C. A., Fontana, P. C., Yates, A. N., Greene, K., Broniatowski, D. A., & Przybocki, D. A. (2021). Four principles of explainable artificial intelligence. National Institute of Standards and Technology. https://doi.org/10.6028/NIST.IR.8312
Piketty, T. (2014). Capital in the twenty-first century. Harvard University Press. https://doi.org/10.4159/9780674369542
Rees, M. (2003). Our final hour: A scientist’s warning – How terror, error, and environmental disaster threaten humankind’s future in this century – On Earth and beyond. Basic Books.
Romer, P. M. (1990). Endogenous technological change. Journal of Political Economy, 98(5). https://doi.org/10.1086/261725
Roodman, D. (2020, November 21). On the probability distribution of long-term changes in the growth rate of the global economy: An outside view. Open Philanthropy. https://www.openphilanthropy.org/sites/default/files/Modeling-the-human-trajectory.pdf
Roser, M. (2022). The future is vast – what does this mean for our own life? Our World in Data. https://ourworldindata.org/the-future-is-vast
Roser, M. (2023). AI timelines: What do experts in artificial intelligence expect for the future? Our World in Data. https://ourworldindata.org/ai-timelines
Russell, S. (2014, November 14). Of myths and moonshine. Reply to: The myth of AI. A conversation with Jaron Lanier. https://www.edge.org/conversation/jaron_lanier-the-myth-of-ai
Sandberg, A., & Bostrom, N. (2008). Global catastrophic risks survey (Technical report #2008-1). Oxford University, Future of Humanity Institute. https://www.fhi.ox.ac.uk/reports/2008-1.pdf
Sevilla, J., Heim, L., Ho, A., Besiroglu, T., Hobbhahn, M., & Villalobos, P. (2022, July 18–23). Compute trends across three eras of machine learning. In Proceedings of the 2022 International Joint Conference on Neural Networks (IJCNN). Padua, Italy. IEEE. https://doi.org/10.1109/IJCNN55064.2022.9891914
Solow, R. M. (1987). We’d better watch out. New York Times Book Review.
Tooby, J., & Cosmides, L. (1992). The psychological foundations of culture. In J. Barkow, L. Cosmides, & J. Tooby (Eds.), The adapted mind: Evolutionary psychology and the generation of culture (pp. 19–136). Oxford Academic. https://doi.org/10.1093/oso/9780195060232.003.0002
Torres, E. P. (2022, September 10). Selling “longtermism”: How PR and marketing drive a controversial new movement. Salon. https://www.salon.com/2022/09/10/selling-longtermism-how-pr-and-marketing-drive-a-controversial-new-movement/
Trammell, P. (2021). Existential risk and exogenous growth. Global Priorities Institute, University of Oxford. https://philiptrammell.com/static/ExistentialRiskAndExogenousGrowth.pdf
Trammell, P., & Korinek, A. (2020). Economic growth under transformative AI (Working Paper No. 8). Global Priorities Institute, University of Oxford. https://globalprioritiesinstitute.org/wp-content/uploads/Philip-Trammell-and-Anton-Korinek_economic-growth-under-transformative-ai.pdf
United Nations. (2022). World population prospects 2022. https://population.un.org/wpp/
Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., Chi, E. H., Hashimoto, T., Vinyals, O., Liang, P., Dean, J., & Fedus, W. (2022). Emergent abilities of large language models. Arxiv: 2206.07682. https://doi.org/10.48550/arXiv.2206.07682
World Economic Forum. (2023). Global cybersecurity outlook 2023 (Insight report). https://www3.weforum.org/docs/WEF_Global_Security_Outlook_Report_2023.pdf
Yudkowsky, E. (2004). Coherent extrapolated volition. The Singularity Institute, San Francisco, CA. https://intelligence.org/files/CEV.pdf
Yudkowsky, E. (2007). Pascal’s mugging: Tiny probabilities of vast utilities. Less Wrong. https://www.lesswrong.com/posts/a5JAiTdytou3Jg749/pascal-s-mugging-tiny-probabilities-of-vast-utilities
Yudkowsky, E. (2008). Artificial intelligence as a positive and negative factor in global risk. In N. Bostrom & M. M. Ćirković (Eds.), Global catastrophic risks (pp. 308–345). Oxford University Press. https://doi.org/10.1093/oso/9780198570509.003.0021
Yudkowsky, E. (2017). There’s no fire alarm for artificial general intelligence. Machine Intelligence Research Institute. https://intelligence.org/2017/10/13/fire-alarm/
Yudkowsky, E. (2022). MIRI announces new “Death with dignity” strategy. Less Wrong. https://www.lesswrong.com/posts/j9Q8bRmwCgXRYAgcJ/miri-announces-new-death-with-dignity-strategy
Acemoglu, D., & Restrepo, P. (2018). The race between man and machine: Implications of technology for growth, factor shares and employment. American Economic Review, 108(6), 1488–1542. https://doi.org/10.1257/aer.20160696
Albanesi, S., Dias da Silva, A., Jimeno, J. F., Lamo, A., & Wabitsch, A. (2023). New technologies and jobs in Europe. (Working Paper No. 31357). National Bureau of Economic Research. https://doi.org/10.3386/w31357
Aschenbrenner, L. (2020). Existential risk and growth (Working Paper No. 6). Columbia University and Global Priorities Institute, University of Oxford. https://globalprioritiesinstitute.org/wp-content/uploads/Leopold-Aschenbrenner_Existential-risk-and-growth_.pdf
Autor, D., Dorn, D., Katz, L., Patterson, C., & Van Reenen, J. (2020). The fall of the labor share and the rise of superstar firms. The Quarterly Journal of Economics, 135(2), 645–709. https://doi.org/10.1093/qje/qjaa004
Barro, R. J. (2003). Determinants of economic growth in a panel of countries. Annals of Economics and Finance, 4, 231–274. https://down.aefweb.net/WorkingPapers/w505.pdf
Bloom, N., Jones, C. I., Van Reenen, J. & Webb, M. (2020). Are ideas getting harder to find? American Economic Review, 110(4), 1104–1144. https://doi.org/10.1257/aer.20180338
Bostrom, N. (2014). Superintelligence: Paths, dangers, strategies. Oxford University Press.
Bostrom, N., Douglas, T., & Sandberg, A. (2016). The unilateralist’s curse and the case for a principle of conformity. Social Epistemology, 30(4), 350–371. https://doi.org/10.1080/02691728.2015.1108373
Branwen, G. (2022, January 2). The scaling hypothesis. https://gwern.net/scaling-hypothesis
Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S. M., Nori, H., Palangi, H., Ribeiro, M. T. & Zhang, Y. (2023). Sparks of artificial general intelligence: Early experiments with GPT-4. ArXiv:2303.12712. https://doi.org/10.48550/arXiv.2303.12712
Chichilnisky, G. (2000). An axiomatic approach to choice under uncertainty with catastrophic risks. Resource and Energy Economics, 22(3), 221–231. https://doi.org/10.1016/S0928-7655(00)00032-4
Chichilnisky, G., Hammond, P. J., & Stern, N. (2020). Fundamental utilitarianism and intergenerational equity with extinction discounting. Social Choice and Welfare, 54, 397–427. https://doi.org/10.1007/s00355-019-01236-z
Cotra, A. (2020). Draft report on AI timelines. AI Alignment Forum. https://www.alignmentforum.org/posts/KrJfoZzpSDpnrv9va/draft-report-on-ai-timelines
Cotra, A. (2022). Two-year update on my personal AI timelines. AI Alignment Forum. https://www.alignmentforum.org/posts/AfH2oPHCApdKicM4m/two-year-update-on-my-personal-ai-timelines
Davidson, T. (2021). Could advanced AI drive explosive economic growth? Open Philanthropy. https://www.openphilanthropy.org/research/could-advanced-ai-drive-explosive-economic-growth/
Davidson, T. (2023) What a compute-centric framework says about takeoff speeds. Open Philanthropy. https://www.openphilanthropy.org/research/what-a-compute-centric-framework-says-about-takeoff-speeds/
Eloundou, T., Manning, S., Mishkin, P., & Rock, D. (2023). GPTs are GPTs: an early look at the labor market impact potential of large language models (Working Paper No. 2303.10130). Arxiv.org. https://doi.org/10.48550/arXiv.2303.10130
Etzioni, O. (2016). No, the experts don’t think superintelligent AI is a threat to humanity. MIT Technology Review. https://www.technologyreview.com/2016/09/20/70131/no-the-experts-dont-think-superintelligent-ai-is-a-threat-to-humanity/
Frey, C. B., & Osborne, M. (2017). The future of employment: How susceptible are jobs to computerisation? Technological Forecasting and Social Change, 114, 254–280. https://doi.org/10.1016/j.techfore.2016.08.019
Gordon, R. J. (2016). The rise and fall of American growth: The U.S. standard of living since the Civil War. Princeton University Press. https://doi.org/10.1515/9781400873302
Grace, K. (2022). Let’s think about slowing down AI. Less Wrong. https://www.lesswrong.com/posts/uFNgRumrDTpBfQGrs/let-s-think-about-slowing-down-ai
Grace, K., Stewart, H., Sandkühler, J. F., Thomas, S., Weinstein-Raun, B., & Brauner, J. (2024). Thousands of AI authors on the future of AI. Arxiv:2401.02843. https://doi.org/10.48550/arXiv.2401.02843
Growiec, J. (2022a). Accelerating economic growth: lessons from 200 000 years of technological progress and human development. Springer. https://doi.org/10.1007/978-3-031-07195-9
Growiec, J. (2022b). Automation, partial and full. Macroeconomic Dynamics, 26(7), 1731–1755. https://doi.org/10.1017/S1365100521000031
Growiec, J. (2023). What will drive global economic growth in the digital age? Studies in Nonlinear Dynamics and Econometrics, 27(3), 335–354. https://doi.org/10.1515/snde-2021-0079
Gruetzemacher, R. & Whittlestone, J. (2021). The transformative potential of artificial intelligence. Arxiv:1912.00747. https://doi.org/10.48550/arXiv.1912.00747
Hanson, R., & Yudkowsky, E. (2013). The Hanson-Yudkowsky AI-foom debate. Machine Intelligence Research Institute. https://intelligence.org/files/AIFoomDebate.pdf
Harari, Y. N. (2014). Sapiens: A brief history of humankind. Vintage.
Hawking, S., Russell, S., Tegmark, M., & Wilczek, F. (2014, May 1). Stephen Hawking: ‘Transcendence looks at the implications of artificial intelligence - but are we taking AI seriously enough?’. Independent. https://www.independent.co.uk/news/science/stephen-hawking-transcendence-looks-at-the-implications-of-artificial-intelligence-but-are-we-taking-ai-seriously-enough-9313474.html
Hendrycks, D. (2023). Natural selection favors AIs over humans. Arxiv: 2303.16200v4. https://doi.org/10.48550/arXiv.2303.16200
Hilbert, M., & López, P. (2011). The world’s technological capacity to store, communicate, and compute information. Science, 332(6205), 60–65. https://doi.org/10.1126/science.1200970
Hilton, B. (2022). Preventing an AI-related catastrophe. 80 000 Hours. https://80000hours.org/problem-profiles/artificial-intelligence/
Johansen, A. & Sornette, D. (2001). Finite-time singularity in the dynamics of the world population, economic and financial indices. Physica A: Statistical Mechanics and its Applications, 294(3–4), 465–502. https://doi.org/10.1016/S0378-4371(01)00105-4
Jones, C. I. (2023). The AI dilemma: Growth versus existential risk (Working Paper No. 31837). National Bureau of Economic Research. https://doi.org/10.3386/w31837
Klump, R., McAdam, P., & Willman, A. (2012). The normalized CES production function: Theory and empirics. Journal of Economic Surveys, 26(5), 769–799. https://doi.org/10.1111/j.1467-6419.2012.00730.x
Korinek, A. (2023). Language models and cognitive automation for economic research. (Working Paper No. 30957). National Bureau of Economic Research. https://doi.org/10.3386/w30957
Korinek, A., & Juelfs, M. (2022). Preparing for the (non-existent?) future of work. In J. B. Bullock, Y.-C. Chen, J. Himmelreich, V. M. Hudson, A. Korinek, M. M. Young, & B. Zhang (Eds.), The Oxford handbook of AI governance. Oxford University Press. https://doi.org/10.1093/oxfordhb/9780197579329.013.44
Kosinski, M. (2023). Theory of mind may have spontaneously emerged in large language models. Arxiv: 2302.02083. https://doi.org/10.48550/arXiv.2302.02083
Krakovna, V., Uesato, J., Mikulik, V., Rahtz, M., Everitt, T., Kumar, R., Kenton, Z., Leike, J., & Legg, S. (2020, April 21). Specification gaming: The flip side of AI ingenuity. DeepMind. https://www.deepmind.com/blog/specification-gaming-the-flip-side-of-ai-ingenuity
Kurzweil, R. (2005). The singularity is near: When humans Transcend biology. Penguin.
Leike, J., & Sutskever, I. (2023, July 5). Introducing superalignment. OpenAI. https://openai.com/blog/introducing-superalignment
Martin, I., & Pindyck, R. S. (2015). Averting catastrophes: The strange economics of Scylla and Charybdis. American Economic Review, 105(10), 2947–2985. https://doi.org/10.1257/aer.20140806
McAskill, W. (2022). What we owe the future: A million-year view. Basic Books.
Milanovic, B. (2016). Global inequality: A new approach for the age of globalization. Harvard University Press. https://doi.org/10.4159/9780674969797
Muehlhauser, L., & Salamon, A. (2012). Intelligence explosion: Evidence and import. In A. Eden, J. Soraker, J. H. Moor, & E. Steinhart (Eds.), Singularity hypotheses: A scientific and philosophical assessment (pp. 15–42). Springer. https://doi.org/10.1007/978-3-642-32560-1_2
Nagy, B., Farmer, J. D., Trancik, J. E., & Gonzales, J. P. (2011). Superexponential long-term trends in information technology. Technological Forecasting and Social Change, 78(8), 1356–1364. https://doi.org/10.1016/j.techfore.2011.07.006
Nordhaus, W. D. (2021). Are we approaching an economic singularity? Information technology and the future of economic growth. American Economic Journal: Macroeconomics, 13(1), 299–332. https://doi.org/10.1257/mac.20170105
Organization for Economic Cooperation and Development. (2024). Real GDP long term forecast. OECD. https://data.oecd.org/gdp/real-gdp-long-term-forecast.htm
OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., Avila, R., Babuschkin, I., Balaji, S., Balcom, V., Baltescu, P., Bao, H., Bavarian, M., Belgum, J. …, Zoph, B. (2023). GPT-4 technical report. Arxiv: 2303.08774. https://doi.org/10.48550/arXiv.2303.08774
Ord, T. (2020). The precipice: Existential risk and the future of humanity. Hachette.
Parfit, D. (1984). Reasons and persons. Oxford University Press. https://www.stafforini.com/docs/Parfit%20-%20Reasons%20and%20persons.pdf
Parteka, A., & Kordalska, A. (2023). Artificial intelligence and productivity: Global evidence from AI patent and bibliometric data. Technovation, 125, Article 102764. https://doi.org/10.1016/j.technovation.2023.102764
Phillips, P. J., Hahn, C. A., Fontana, P. C., Yates, A. N., Greene, K., Broniatowski, D. A., & Przybocki, D. A. (2021). Four principles of explainable artificial intelligence. National Institute of Standards and Technology. https://doi.org/10.6028/NIST.IR.8312
Piketty, T. (2014). Capital in the twenty-first century. Harvard University Press. https://doi.org/10.4159/9780674369542
Rees, M. (2003). Our final hour: A scientist’s warning – How terror, error, and environmental disaster threaten humankind’s future in this century – On Earth and beyond. Basic Books.
Romer, P. M. (1990). Endogenous technological change. Journal of Political Economy, 98(5). https://doi.org/10.1086/261725
Roodman, D. (2020, November 21). On the probability distribution of long-term changes in the growth rate of the global economy: An outside view. Open Philanthropy. https://www.openphilanthropy.org/sites/default/files/Modeling-the-human-trajectory.pdf
Roser, M. (2022). The future is vast – what does this mean for our own life? Our World in Data. https://ourworldindata.org/the-future-is-vast
Roser, M. (2023). AI timelines: What do experts in artificial intelligence expect for the future? Our World in Data. https://ourworldindata.org/ai-timelines
Russell, S. (2014, November 14). Of myths and moonshine. Reply to: The myth of AI. A conversation with Jaron Lanier. https://www.edge.org/conversation/jaron_lanier-the-myth-of-ai
Sandberg, A., & Bostrom, N. (2008). Global catastrophic risks survey (Technical report #2008-1). Oxford University, Future of Humanity Institute. https://www.fhi.ox.ac.uk/reports/2008-1.pdf
Sevilla, J., Heim, L., Ho, A., Besiroglu, T., Hobbhahn, M., & Villalobos, P. (2022, July 18–23). Compute trends across three eras of machine learning. In Proceedings of the 2022 International Joint Conference on Neural Networks (IJCNN). Padua, Italy. IEEE. https://doi.org/10.1109/IJCNN55064.2022.9891914
Solow, R. M. (1987). We’d better watch out. New York Times Book Review.
Tooby, J., & Cosmides, L. (1992). The psychological foundations of culture. In J. Barkow, L. Cosmides, & J. Tooby (Eds.), The adapted mind: Evolutionary psychology and the generation of culture (pp. 19–136). Oxford Academic. https://doi.org/10.1093/oso/9780195060232.003.0002
Torres, E. P. (2022, September 10). Selling “longtermism”: How PR and marketing drive a controversial new movement. Salon. https://www.salon.com/2022/09/10/selling-longtermism-how-pr-and-marketing-drive-a-controversial-new-movement/
Trammell, P. (2021). Existential risk and exogenous growth. Global Priorities Institute, University of Oxford. https://philiptrammell.com/static/ExistentialRiskAndExogenousGrowth.pdf
Trammell, P., & Korinek, A. (2020). Economic growth under transformative AI (Working Paper No. 8). Global Priorities Institute, University of Oxford. https://globalprioritiesinstitute.org/wp-content/uploads/Philip-Trammell-and-Anton-Korinek_economic-growth-under-transformative-ai.pdf
United Nations. (2022). World population prospects 2022. https://population.un.org/wpp/
Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., Chi, E. H., Hashimoto, T., Vinyals, O., Liang, P., Dean, J., & Fedus, W. (2022). Emergent abilities of large language models. Arxiv: 2206.07682. https://doi.org/10.48550/arXiv.2206.07682
World Economic Forum. (2023). Global cybersecurity outlook 2023 (Insight report). https://www3.weforum.org/docs/WEF_Global_Security_Outlook_Report_2023.pdf
Yudkowsky, E. (2004). Coherent extrapolated volition. The Singularity Institute, San Francisco, CA. https://intelligence.org/files/CEV.pdf
Yudkowsky, E. (2007). Pascal’s mugging: Tiny probabilities of vast utilities. Less Wrong. https://www.lesswrong.com/posts/a5JAiTdytou3Jg749/pascal-s-mugging-tiny-probabilities-of-vast-utilities
Yudkowsky, E. (2008). Artificial intelligence as a positive and negative factor in global risk. In N. Bostrom & M. M. Ćirković (Eds.), Global catastrophic risks (pp. 308–345). Oxford University Press. https://doi.org/10.1093/oso/9780198570509.003.0021
Yudkowsky, E. (2017). There’s no fire alarm for artificial general intelligence. Machine Intelligence Research Institute. https://intelligence.org/2017/10/13/fire-alarm/
Yudkowsky, E. (2022). MIRI announces new “Death with dignity” strategy. Less Wrong. https://www.lesswrong.com/posts/j9Q8bRmwCgXRYAgcJ/miri-announces-new-death-with-dignity-strategy