'AI godfather' worries we're creating 'alien beings' with artificial intelligence, with pesky moral implications

Geoffrey Hinton is a computer scientist known for his foundational work in AI, specifically when it comes to neural networks (ie, approaching machine-learning a bit like our fleshy brain-learning). As such, he's been described as the 'AI godfather'. But, perhaps surprisingly, he recently described AI as "alien beings" that humans have a "moral obligation" towards.

"We’re creating alien beings—meaning we’re not the only intelligent beings around," He said in a recent interview with The Guardian, "And the reality is, we only have a couple of years to adapt to that.”

Hinton claims, "AI models understand what they’re saying. You couldn’t possibly answer all the questions without understanding the question. It’s ridiculous to say they don’t understand.”

Now, far be it for me to argue with an expert in the field, but I feel like 'understanding' is doing a lot of heavy lifting here. For just one thing, I'm not sure an 'answer' consisting of what is statistically most likely to follow a prompted question is really 'understanding.' In other words, I don't have a Nobel Prize laureate like Hinton does, but I am at least familiar with the Chinese Room thought experiment.

Hinton also says, "The reality is, we’re making beings of a different kind. And the big tech companies keep saying: we want it smarter. This is crazy. We should care as much about all the other aspects of a being as you do about its intelligence.”

AI regulation and 'alignment' (ie, training AI to have the same goals and values as humans) have been hot button issues lately. For instance, the OpenAI-Hugging Face incident is not yet a small dot in the rearview mirror, and the president of Microsoft claims that a carton of milk is more regulated than AI.

Hinton cut his ties to Google in 2023 in order to "freely speak out about the risks of AI," and he's sceptical about the AI industry's attempts to self-regulate. He quips, "Do we think tobacco companies should have been allowed to design the warnings on the packets? So that they could print them in a six-point font, in dark grey on black?"

Even so, Hinton also expresses concerns about the path to AI regulation, saying that humans bear a moral obligation towards AI models.

"Once you start talking about, ‘We have a moral obligation towards these alien beings,’ you lose your audience. I’ve struggled with this issue," Hinton says, "We care about people. [But] we’re making these things and there’s nothing to stop us giving these things—which are more intelligent than us—less rights than us. I’m willing to be selfish and choose human beings. Of course, we should try to make them so that they’re very happy in a symbiotic relationship with us, where our concerns are weighed more heavily than their concerns."

Hinton argues, “People have a very bad model of what people are, and that’s why they fail to understand how these AI models are very like us. Most people don’t understand what consciousness is. They don’t understand what subjective experience is, and they’re very confident that even though they don’t understand it, they’re sure AI doesn’t have it.”

While I'm not a doctor of Philosophy myself, I do know someone who is—our very own Jacob Fox. He's already shared his thoughts on the subject of 'AI consciousness' here, and I asked him for his insight again on this story.

Jacob told me he did not agree with Hinton's moral obligation argument, going on to write more broadly, "While it's good that Hinton is acknowledging subjective experience is relevant to consciousness, he hasn't given a single good reason why we should take seriously the idea that AI has it. And when we interpret personhood as to do with objective characteristics such as rationality, behaviour, and so on, rather than experience, we run the risk of ascribing moral and legal and political weight to them. Which runs the risk of giving AI companies a massive moral argument to keep the industry going, despite how it affects actual humans."

To put it another way, the industry argument for AI consciousness or AI rights is not an altruistic one—it is arguably a cynical tactic to continue business as usual.