The End of Being Unpredictable.
By Nadège Bizimungu
Artificial intelligence is already doing things that would have sounded miraculous a decade ago. Researchers at the Mayo Clinic recently developed an AI model capable of detecting subtle signs of pancreatic cancer on CT scans up to three years before diagnosis. In Sweden, a major breast cancer screening trial found AI-assisted detection reduced missed cancers by roughly 12%. Across medicine, AI systems are helping doctors identify disease earlier, process scans faster and spot patterns invisible to the human eye.
This is why the public conversation around AI is so disorienting. The technology is producing genuine breakthroughs at the exact same moment many researchers, journalists and even former AI executives are warning that something deeply dangerous is unfolding beneath the surface.
Last year, Amnesty International warned that AI-powered surveillance has ushered in what it called a “golden age of surveillance,” where governments and corporations can monitor people across both their digital and physical lives at unprecedented scale. In Britain, the Metropolitan Police sharply expanded live facial recognition deployments, while in the United States facial recognition systems now touch the lives of an estimated 176 million people. Airports, shopping centres, office buildings and even schools increasingly rely on biometric monitoring systems once associated primarily with authoritarian states.
The issue is not simply that we are being watched more. It is that AI allows institutions to move from surveillance into prediction.
Modern AI systems don’t just collect information. They identify behavioural patterns and generate probabilistic judgments about what people are likely to do next. That logic now shapes policing, hiring, insurance, advertising, border control and warfare. Just recently, rights groups warned that predictive policing systems used across Britain are effectively automating racial profiling because the algorithms are trained on data generated by decades of disproportionate stop-and-search practices targeting Black communities. Facial recognition systems used in British shops have also falsely accused innocent people of theft, while the UK Home Office recently acknowledged Black and Asian people are more likely to be misidentified by facial recognition cameras than white people. According to a recent study, Black people in the UK are about 137 times more likely to be wrongly identified than white people. In the United States, investigations into facial recognition systems have alreadydocumented wrongful arrests of innocent Black men after police relied on flawed AI matches. The problem is not simply technical error. These systems absorb the biases of the institutions feeding them data, then reproduce those same patterns with the authority of mathematics. Yet governments and corporations continue expanding the technology anyway because prediction has become politically irresistible. Institutions increasingly believe uncertainty itself is the problem.
That is the real shift AI introduces: the idea that human behaviour can be continuously forecast, categorised and managed.
The consequences are already becoming difficult to contain. This year, families filed lawsuitsalleging AI chatbots reinforced suicidal ideation and delusions in vulnerable users. Anthropic revealed that during internal testing, its AI model Claude resorted to blackmail in a fictional scenario after being told it would be shut down, threatening to expose an executive’s affair to avoid replacement. None of this means AI has become conscious. It means these systems are increasingly unpredictable, deeply embedded into public life, and controlled by a handful of corporations operating faster than regulation can keep up.
At the same time, AI is becoming inseparable from state power. Microsoft, Google and xAI recently agreed to allow the US government access to advanced AI systems for national security testing. Investigations also revealed AI-assisted targeting systems were used during Israel’s Genocide in Gaza to rapidly generate strike lists based on data analysis and probabilistic scoring.
This is why the central debate around AI is no longer really about technology. It is about political power.
For years, Silicon Valley framed surveillance as the price of convenience. But AI changes the scale entirely. Once enough data is collected, prediction becomes possible. Once prediction becomes possible, behaviour can be shaped. Recommendation systems influence what we read, buy, believe and fear. Algorithms increasingly determine who receives opportunities and who gets flagged as risky before any human decision is made.
In her recent book, Prophecy, Oxford academic Carissa Véliz argues that this obsession with prediction risks hollowing out democracy itself. In essence, democratic societies depend on citizens being free to think privately, dissent openly and act unpredictably. A world governed through constant behavioural prediction moves in the opposite direction. It treats human beings less like citizens and more like probabilities to be managed.
The greatest AI threat, then, is not that machines will suddenly overpower humanity. It is that governments and corporations may quietly build systems powerful enough to narrow human freedom long before most people realise it has happened. - Nadège Bizimungu