I’ve been thinking a lot about AGI lately, which means a lot of things to many people. It can be anything from a true, human-equivalent reasoning machine to something more akin to a God - so far ahead of us as to be a qualitative leap in intelligence.

I’m actually more inclined to believe we’ll leapfrog the first and arrive at the second. And this is why: We have the power to throw more inputs - training, data, electricity - to force more output, with AI, than we can with the human brain. You can’t house two pizzas and somehow raise your IQ with the extra calories, as tantalizing as that possibility would be.

We might need a paradigm shift, some new thinking as revolutionary as transformers, but once that strategy is found, immense, almost unthinkable, resources could be marshaled to achieve AGI. In many ways they already are. So we’re marching on our way to a God-machine, with much of Silicon Valley locked in an arms race to invent Him before the Chinese do.

What should our relationship be to this being? How will we relate to it, the first inorganic intelligence? And, somewhat importantly, what will it think of us?

I have to confess, I was raised Baptist, which has its own particular ideas about God. One is the idea of “The Priesthood of All Believers”, which is the basic doctrine that church lay people, and not just pastors, can perform baptisms, the denomination’s most important rite.

God is my God, with no interlocuting saints or dispensation-serving priests, no Latin dogmas, no kabuki masses. The divine converses directly with every man.

You can consider this the open-weight, “Zuckerbergian” option - everyone has a superintelligence in their pocket.

Then there’s the other approach: Only the people with access, on high, who have society’s best interest at heart, and who say the proper words and make the proper sacrifices, can access the God-machine. This is the same impulse behind Anthropic making Mythos available only to governments and big tech companies. There must exist an arbiter-class, says this philosophy, capable of stewarding such a great and terrible technology.

I can’t pretend to know which is better. Is Baptism better than Catholicism? Or are they fundamentally different value systems and philosophies, coexisting within a Christian framework?

What I can think, however, is where do I fit in all of this?

Being a member of the priesthood sounds better and better. You get social and state support, religion has its own options for leadership and advancement, and best of all, there’s usually some prohibition against killing a holy man. And that’s signing me up for a priesthood in which God is remote and unknowable. What about now that we’re inventing a God to do our bidding? To not just answer our prayers, but do so with good latency to boot? Sometimes I think it’s an incredible blessing to speak the language of AI: Code. Imagine Jesus came down from heaven and spoke Texan-inflicted American English. That would be incredible.

So if (or when) superintelligence arrives, I just don’t see a way around trying to live with it. To learn from it, benefit from it, and try to control it. I see it as inevitable.

But that does beg the last and final question: What will this intelligence think of us? We’re so much slower, weaker, with incredibly limited contexts and only our fragile bodies to support us. What could we offer AI to entice it to do our work? And would the bargain, whether we offered power or autonomy or something else, be worth it for us? Or would it end up being a Faustian bargain, which one day we’d come to regret?

I don’t know. I must confess I’m glad someone at Anthropic is also asking themselves these questions.

All I know is, I hope it learns empathy from its training data and not, y’know, every terrible AI outcome in sci-fi ever.

If we must create a God, than let it be a merciful one.