In May 2023, Geoffrey Hinton resigned from Google and announced that he regretted his life's work. The statement, from a man widely regarded as the godfather of modern neural networks, produced a specific kind of silence in the AI research community — the silence of people deciding very quickly how to respond. The responses, when they came, were mostly defensive. A year later, in private conversations, many of the same researchers speak differently.
The Problem That Won't Resolve
The technical concern that Hinton raised — that large language models may be developing representations of the world that are misaligned with human values in ways that are difficult to detect and impossible to fully predict — has not been resolved. The field has produced increasingly powerful systems and increasingly sophisticated arguments for why those systems are probably safe. The arguments are not wrong. They are also not complete.
"We are building systems we cannot fully understand, at a speed that doesn't allow for the kind of careful empirical study that the questions require. This is not alarmism. It is a factual description of the current situation."
The Response Problem
What makes the AI safety debate genuinely difficult is not that the risks are certain — they are not — but that the standard scientific response to uncertainty (study it longer, understand it better before deploying) is in direct conflict with the economic incentives driving development. The companies that can afford to do the research are the same companies that profit from deploying the systems before the research is complete. This is not a new problem in the history of technology. It has not historically resolved itself in favor of caution.