The audacity of building AGI

A personal reflection

My all-things-considered view on AGI is that it could be massively good, and it would probably be a huge moral loss if humanity failed to build powerful artificial intelligence. The current world has far too much suffering, and I want this suffering to end. I also expect AGI to cause a lot of harm to current people, even if we avoid extinction-level catastrophe. This post is an attempt to give space to myself and an emotional reaction I have to the development of AGI — it is only a sliver of that all-things-considered view. 

—

I’m 23 years old. Sometimes I have glimpses of a normal future. A future where I go to grad school, become a professor, teach classes to interested students, host coffee conversations where we discuss worldly events and philosophy. Or a future where I fall in love, cultivate an amazing relationship over the course of many years, build a decades-long anthology of memories and inside jokes. 

I guess it’s normal to dream about these things, to think about one’s future, the love, adventure, passion. I guess a lot of people hope for the future in this way. I guess a lot of people have plans for the rest of their life. Plans that involve the world looking similar to the way it does now. Plans that involve living into old age. Plans that involve humans having jobs. Plans that involve stability in the world. 

But some assholes in San Francisco are going to ruin it. They’re going to ruin everything. Some motherfucking assholes who are really good at coding are going to flip the whole world upside down, whether you like it or not. There are like 10,000 of these people, like 10,000 people who are really pushing progress on building the god machine. And they’re gonna change the whole goddamn world. 

They don’t have your permission, or anybody’s. Apparently you don’t need anybody’s permission to make a god machine. 

—

A couple asides: 

I find it somewhat ironic that these assholes are often libertarian-ish. That the doctrine of individual liberties gives these assholes the self-justification to impose a world-flipping god machine on eight billion of their peers. “I like your Christ, I do not like your Christians. Your Christians are so unlike your Christ.”

The creation of AGI will be the most heinous act of imperialism ever carried out. California’s largest export, currently “Computer and Electronic Product Manufacturing”, will be mass death, or perhaps merely the unemployment of like 99% of the world. It is such a strong act of injustice to build a god machine. 

— 

I feel like my dreams are being ripped away. And all the life plans that the billions of other people also have — ripped away. So I suppress the glimpses. I don’t let them develop into anything more. I won’t let myself get invested in a counterfactual future of love and joy and humanness. Because we live in a timeline where a few thousand assholes can destroy the world and kill everybody I have ever loved and everybody I would have ever loved and everything human. 

In such a timeline, if you can, you work to make things better. The god machine does not wait a few decades for me to live out my dreams of professoring. 

—

I do not know where the consequentialist chips will fall at the end of this. But I do think that those contributing to the development of powerful artificial intelligence right now are doing a thing which has a very high likelihood of being deeply unjust, authoritarian, and evil.