Thoughts
What happens when hostile nations use AI to weaponise our personal data and destabilise society?
Zombie films have always understood something about the apocalypse that science fiction about artificial intelligence often overlooks.
The monsters are frightening, but they’re rarely the greatest threat. It’s what people do to one another when the rules change.
We’ve spent decades imagining an AI apocalypse in which machines become conscious and turn against humanity. Skynet, killer robots, the machines deciding we’re surplus to requirements.
But what if the machines never turn against us?
What if we use them to turn against each other?
What if this is it?
I’ve been watching the succession of outages affecting digital services and infrastructure in the United States.
Payment systems, cloud services, communications networks. Even reports of disruption involving national security infrastructure.
There’s nothing particularly compelling to suggest these incidents are connected. Technology fails all the time. Cloudflare has outages. Apple Pay has problems. Networks go down, software updates break things and infrastructure is considerably more fragile than we’d like to believe.
Coincidence remains a perfectly reasonable explanation.
But I keep coming back to a question.
What if this is what the beginning of AI-enabled cyberwarfare looks like?
Not a spectacular attack that brings down an entire country overnight, but a succession of incidents that gradually undermine confidence in the systems we depend on.
A payment system stops working. A communications network goes offline. An essential service becomes unavailable.
Individually, each incident might be explained away as another technical failure.
Collectively, if they were deliberate, they could create uncertainty about whether anything is safe.
I’m not suggesting that’s what’s happening in America. We don’t have the evidence to make that claim.
But the possibility feels less like science fiction than it did even a year ago.
And that’s because AI is changing what’s possible.
The impossible is becoming possible
Cyberwarfare isn’t new.
Governments have spent decades developing capabilities to penetrate networks, steal intelligence, disrupt infrastructure and interfere with their adversaries.
The difference is that these operations have traditionally required considerable expertise, resources and time.
AI is beginning to change those constraints.
In 2025, Anthropic reported disrupting a cyber-espionage campaign it attributed to a Chinese state-sponsored group. According to the company’s investigation, AI agents carried out an estimated 80–90% of the tactical work involved in the operation.
That included reconnaissance, vulnerability identification, exploitation and analysing information obtained from compromised systems.
The assessment came from Anthropic itself, and it doesn’t mean AI can independently compromise any system it chooses. But it demonstrates how far the technology has progressed beyond generating snippets of malicious code.
We’re beginning to see AI participate in substantial parts of cyber operations.
And that raises an uncomfortable question.
If these are the capabilities being detected and publicly reported, what might the most sophisticated intelligence organisations be developing behind closed doors?
We don’t know.
But we do know that nation-states have motivations and resources that extend far beyond those of ordinary criminal organisations.
Countries such as Iran, China, Russia and North Korea have established cyber capabilities. Their objectives can include espionage, disruption, political influence and undermining confidence in rival governments.
AI doesn’t need to invent those objectives.
It just needs to make pursuing them easier.
You don’t need to defeat a country to destabilise it
When we imagine a cyberattack against a nation, we tend to think about critical infrastructure.
Power stations, banking systems, telecommunications, transport networks and government departments.
Those are obvious targets because disrupting them can have enormous consequences.
But there’s another possibility.
What if, instead of attacking the systems that support a population, a hostile state could attack the population itself?
Not physically.
Personally.
Imagine a government obtaining an enormous collection of private information about millions of people in another country.
That information might come from compromised services, stolen databases, exposed records or data accumulated through years of breaches.
Until relatively recently, turning that information into individually targeted pressure would have been an enormous undertaking.
Someone would have needed to examine the records, identify useful information, establish relationships and work out which individuals might be vulnerable to intimidation.
Doing that for a handful of important people is one thing.
Doing it for millions is something else entirely.
But what happens when AI can perform much of that analysis?
What happens when it can identify potentially sensitive information, connect it to individuals and generate convincing communications tailored to their circumstances?
What happens when the threats can be delivered through email, text messages, telephone calls and social media, with much of the process automated?
What happens when personalised intimidation becomes possible at industrial scale?
We already have many of the individual technologies required for such a campaign.
That doesn’t establish that a hostile state has successfully combined them into an operation targeting millions of people. But it’s a scenario that deserves serious consideration.
And unlike conventional cybercrime, a state-sponsored operation wouldn’t necessarily need to make money.
Its purpose could simply be to create fear, confusion and distrust.
I’ve got nothing to hide
For years, whenever conversations turn to online privacy, somebody inevitably says the same thing.
“I’ve got nothing to hide.”
I’ve never found that particularly convincing.
Privacy isn’t about whether you’ve done something wrong. It’s about deciding which parts of your life belong to you.
Think about everything you’ve entrusted to digital services.
Your private conversations. Your relationships. Your finances. Your medical concerns. Your photographs. Your browsing history. Your personal problems.
The things you’ve told friends in confidence.
The things you’ve said when you were angry.
The things you’ve searched for because you were worried, embarrassed or simply curious.
None of it needs to be illegal or immoral.
But would you be comfortable with somebody else having access to all of it?
More importantly, would you be comfortable with somebody deliberately looking for ways to use it against you?
Having nothing to hide isn’t the same as having nothing that can be used against you.
And that’s where AI changes the equation.
A hostile actor doesn’t necessarily need to find evidence of wrongdoing.
They might only need to identify something you care about protecting.
Your reputation. Your relationships. Your employment. Your family.
They might not even need genuine evidence. Generative AI can produce convincing impersonations and fabricated material, creating another opportunity for intimidation.
Imagine receiving a message containing enough genuine personal information to convince you that somebody has access to your private life.
Now imagine being told that information will be sent to your employer, your partner or your friends.
Whether the threat is genuine, exaggerated or entirely fabricated, the fear it creates could be very real.
And imagine that happening to hundreds of thousands of people at once.
What if somebody gets into the AI?
There’s another possibility that makes this particularly uncomfortable.
We’re increasingly trusting AI assistants with information we might never have shared with a conventional website.
We ask them questions about our relationships, our finances, our health and our careers.
We upload documents, discuss problems and work through decisions.
The conversations can be remarkably personal.
Now imagine a hostile state successfully compromising a major AI provider and gaining access to sensitive conversation histories.
I’m not suggesting that ChatGPT or another major AI service has suffered such a breach. There’s no evidence in the incidents discussed here that this has happened.
But the potential consequences deserve consideration.
A sufficiently large collection of private conversations could offer an extraordinary picture of people’s lives.
And the same technology that makes AI useful for understanding those conversations could potentially be used to analyse stolen information.
An attacker wouldn’t necessarily need to read millions of conversations individually.
AI could help organise the information, identify patterns and prioritise material.
The technology that makes these services so useful could also make the consequences of a major breach more serious.
Of course, access controls, encryption, retention policies and security protections matter enormously. A hypothetical breach doesn’t mean every conversation would automatically be exposed.
But it raises a question that I think we should all be asking.
How much of ourselves are we comfortable placing in systems we don’t control?
The apocalypse doesn’t need a consciousness
There’s something almost reassuring about the traditional AI apocalypse.
The machines become conscious. They develop their own objectives. They decide humanity is a problem.
At least we’d know who the enemy was.
The scenario I’m describing is much less straightforward.
AI doesn’t need to become sentient.
It doesn’t need to develop ambitions, hatred or a desire for power.
It can remain exactly what we intended it to be: an extraordinarily capable tool.
The danger comes from the people directing it.
And I think that’s something we’re struggling to come to terms with.
We’re fascinated by what AI might become in the future, while its existing capabilities are already changing the balance of power between people, organisations and governments.
The same technology that allows me to build software at a speed I couldn’t have imagined a few years ago could help a hostile actor undertake work that previously required an enormous team.
That doesn’t make every AI system dangerous or every application malicious.
But it does mean we need to think seriously about what happens when increasingly capable tools are placed in the hands of people who intend to cause harm.
Privacy is becoming a matter of personal security
I’ve spent much of my career building websites and digital products.
Like most people in the industry, I’ve watched the web become increasingly dependent on collecting, storing and analysing personal information.
We’ve encouraged people to move their lives online because it’s convenient.
Store your photographs. Sync your contacts. Back up your messages. Connect your accounts. Tell us your preferences.
And we’ve built enormous businesses around knowing as much as possible about people.
For a long time, the privacy debate has focused on advertising, tracking and surveillance.
Those are important issues, but I think AI introduces another dimension.
Personal information isn’t simply something that can be collected and monetised. It’s something that can potentially be weaponised.
That makes data minimisation, sensible retention policies, encryption and secure systems more important than ever.
It also makes me question how casually we surrender information about ourselves.
We can’t eliminate the risk. Much of our data is already scattered across systems we have no direct control over.
But we can be more deliberate about what we share, what we retain and which services we trust.
And those of us building digital products have a responsibility not to collect information simply because we can.
Every unnecessary piece of personal data becomes another potential liability.
The danger might already be human
I don’t know whether the recent outages in America are connected.
I haven’t seen compelling evidence that they are, and I wouldn’t want to contribute to the confusion by suggesting otherwise.
But they have made me think about how dependent we’ve become on digital infrastructure, and how quickly confidence in that infrastructure could disappear.
More importantly, they’ve made me think about what happens when AI makes sophisticated cyber operations easier to conduct.
We’ve spent decades imagining the end of the world at the hands of intelligent machines.
But perhaps the more immediate danger isn’t artificial intelligence developing a will of its own.
Perhaps it’s governments using increasingly powerful AI to disrupt societies, undermine institutions and exploit the private lives of ordinary people.
Not because those people have done anything wrong.
Not because they’re important.
But because technology could make targeting them individually, at enormous scale, possible.
We thought the danger was AI learning everything about us. Perhaps the danger is what happens when somebody else asks it what to do with that information.