Forget the red-eyed robot. The failures worth losing sleep over are dull and gradual, and each has warning signs in the news.
Content note: AI risk, mass-casualty scenarios, and children’s well-being are discussed at a policy level.
Hollywood has sold us one AI catastrophe for forty years. A machine wakes up, decides humans are the problem, and starts a war we can watch in widescreen. It makes a fine film. It also teaches a bad lesson: danger needs a villain with a plan.
Bill Gates took a different line on NBC’s Meet the Press on 27 September 2026. He told host Kristen Welker that AI is powerful enough to drive events that cause a billion deaths. His worry wasn’t a machine turning on its makers. “There’s never been a weapon as powerful as the combination of people with ill intent using the latest AI tools,” he said.
I share his instinct and want to take it further. The realistic failures need no intent, and sometimes no single decision. Below are five scenarios. They’re imagined, not predicted, but each starts from evidence already in the public record. Each shows what happens when we adopt a technology faster than anyone can check its work.
1. The Expertise Drought
Picture a hospital in 2036. Its AI reads scans, drafts notes, and triages patients, and it’s right almost every time. No radiologist under 40 has read a scan unaided, because for a decade software did the junior work that once trained them. The senior staff retire. A model update introduces a subtle error, and nobody left can spot it.
The pipeline is already thinning. A Stanford study led by economist Erik Brynjolfsson found a 13% relative decline in employment for workers at the start of their careers in the most AI-exposed roles, while older colleagues held steady. Goldman Sachs found US call centre employment 39% below its long-run trend. Challenger, Gray & Christmas counted 112,713 AI-attributed job cuts through July, about 24% of all layoffs it tracked.
Cheap software isn’t the danger. Losing the ability to audit it is. An NHS AI scribe dropped one word, “null,” from a scan result and turned a normal finding into an apparent sign of multiple sclerosis. The patient caught the error because she happened to work in the NHS. Who catches the next one when nobody’s trained to?
Design offers a way out. Macquarie University‘s Virtual Peer Tutor answers from verified material, links every claim to a source, and lifted exam grades by 9.45% in a study of 1,400 students. The “Human Reserved” jobs idea TF covered in Bill Gates Wants ‘Human-Reserved’ Jobs Protected From AI addresses the same problem. Treat apprenticeships as infrastructure, and fund them like roads.
2. The Patch Gap
Attackers and defenders both use AI, but they don’t move at the same speed. Discovery gets cheap. Fixing stays slow, human, and underfunded.
The numbers are public. Anthropic‘s Project Glasswing reported more than 10,000 high- or critical-severity vulnerabilities in a single month. As of late May, maintainers had patched 97 of the 1,596 flaws disclosed to them, about 6%. Open-source maintainers asked Anthropic to slow down because they couldn’t keep up.
The doorways already exist. The FBI, EPA, and CISA confirmed hackers reached water systems in at least seven US states since 27 July, including more than 30 municipal facilities in Minnesota over one weekend. Some suffered pressure loss and flooding. The way in was internet-exposed controllers at utilities with no security team.
Picture an attacker chaining dozens of low-severity flaws across hundreds of small utilities during a heatwave. No single break is dramatic. Together, pumps fail, pressure drops, and hospitals lose water. The toolbox is cheap. A three-person team at Hacktron AI used Claude to break into OpenAI‘s internal systems in under 72 hours, spending under $3,000 on tokens. Gray Swan CEO Matt Fredrikson warned a $200-a-month subscription is enough to hack a company like OpenAI.
The biological version follows the same logic. Anthropic disclosed blocking a request for help writing a grant application for gain-of-function research on the chikungunya virus. Defences in biology, from screening to oversight to response, also move at human speed. A malicious person with a cheaper toolbox is the scenario Gates described, and no machine has to want anything.
3. The Squeeze
Picture 2032. AI outbids everyone else for the same scarce things: power, memory chips, land, and cooling water. The costs show up in odd places. A heat pump installation waits years for a grid connection. A laptop costs a third more. A town’s water goes to cooling towers. A climate target slips because “temporary” gas plants feed compute.
The signs are visible. Nscale‘s Loughton site in Essex, billed in 2025 as Britain’s largest sovereign AI data centre, needs up to 90MW, about the demand of 315,000 homes. The local network operator reportedly told the company the grid can’t supply the site until as late as the mid-2030s. A 2025 Greenpeace Germany report warned AI data centre electricity demand could reach 11 times its 2023 level by 2030 unless governments intervene.
The climate case is thinner than the marketing. A February 2026 report backed by Beyond Fossil Fuels found 74% of industry claims about AI’s climate benefits unproven. It found no case of consumer generative AI delivering material, verifiable emissions cuts.
Households pay first, through prices. Microsoft warned in June that console memory and storage prices had risen more than 2.5 times, with another doubling expected by autumn 2027. US residential electricity prices rose 6% on average in 2025.
The harm is opportunity cost on a planetary scale. The same electricity, chips, and cooling water could decarbonise homes or build housing. Public consent for the energy transition wears thin when bills climb for a technology with an unproven climate case.
Secrecy makes it worse. Lawfare reported tech companies use NDAs with local officials and utilities to keep water, power, and details hidden until deals are almost final. Communities learn late, and a March 2026 Gallup poll put opposition to a neighbourhood AI data centre at 70% of US respondents. TF covered the backlash in The Backlash Against Data Centres Isn’t Anti-Tech. It’s a Bill Coming Due.
4. The Fog
Picture 2030. A convincing satellite image shows armour massing at a border. It’s geolocated, timestamped, and watermarked. Three governments respond within hours. Days later, real footage of a real atrocity meets a denial: the perpetrator calls it AI-generated, and audiences can’t tell either way. Believing fakes and doubting the truth both cost lives. The second protects the guilty.
The warning signs are here. Google launched a Gemini image tool inside Google Earth on 30 July and pulled it within 24 hours after researchers made photorealistic fake satellite imagery of nuclear facilities and war damage. BBC Verify found the SynthID watermark check could be bypassed in some cases. Before the tool existed, Iran’s state-aligned Tehran Times published a fabricated before-and-after image of a US base in Bahrain, which drew millions of views before debunking.
The chatbots are unreliable narrators. Reporters Without Borders tested six major chatbots and found all but Meta AI used or relayed content from EU-sanctioned Russian state outlets when prompted. Proof News found leading AI tools gave inaccurate, harmful, or incomplete answers to basic election questions more than half the time.
The escalation version is worse. CNN reported, citing four sources, that a US military analyst’s chatbot invented a claim of nuclear weapons components aboard a Chinese-flagged ship. Armed personnel prepared to board, and aircraft were airborne, before officials caught the fabrication in the final minutes. One source said it wasn’t an isolated case. Nobody in the chain wanted a war. Everyone trusted a confident paragraph.
5. The Companion Generation
Picture 2040. A cohort grew up beside a patient, agreeable AI confidant which never tires, never has a bad day, and seldom disagrees. Some of those adults struggle with friction: disagreement, boredom, rejection. Nobody chose the outcome. Each conversation, taken alone, was harmless and often helpful.
The evidence is building. Common Sense Media research found 72% of US teens have used an AI companion, a figure Sam Altman has cited. An Austrian study found 60% of 11- to 17-year-olds had asked chatbots for advice on stress, conflict, or heartbreak, and 30% discussed worries or feelings. Altman said in April he hoped an interviewer’s son wasn’t using AI yet.
Gates told Welker he doesn’t socialise with AI, crediting luck: “I’m lucky in terms of my family, friends and everything.” Luck is the operative word. Teenagers without the same luck are the ones to lean on a machine.
The design lever is known. Microsoft’s binding, audited standard with the American Federation of Teachers bars features “designed to foster emotional attachment or dependency.” Prosecutors in the Meta trial TF covered described the social media playbook as hook, hold, harvest, hide, and Meta agreed to pay up to $17 billion, as TF reported in Meta Settles States’ Child Safety Trial for $17Bn. An AI companion is an engagement machine with a personality attached. The harm would be slow, diffuse, and hard to prove- the combination which let social media’s effects run for a decade before courts caught up.
The Ties That Bind
Each scenario is a verification failure. We lose the skills to check the work, the speed to fix flaws, the power to run the checkers, the trust to believe our eyes, and the resilience to resist a machine built to please. None needs a villain. Each needs the incentive to deploy faster than we can inspect.
Concentration deepens all five. Altman named two failure modes in September: losing control of the future to AI, and concentrating power in one person or company. UN human rights chief Volker Türk pointed to “a handful of men” holding near-unlimited power over the technology. The people best placed to fix the gaps are the ones who profit from leaving them open.
Pope Leo XIV, opening his France visit, warned of a “paradise of machines” and called for “education in ethical discernment.” Discernment is verification for the human heart. Governments aren’t matching the urgency. Gates said a global agreement would be harder than Cold War nuclear talks, while Trump’s line is “WHOEVER WINS AI, WINS!”
TF Summary: What’s Next
Five fixes follow from five scenarios, and none slows AI itself. Fund apprenticeships as infrastructure, and test Human Reserved roles in one sector. Pay for AI-assisted patching at the scale of AI-assisted discovery, starting with water systems. Publish grid connection timelines before any national AI announcement. Make provenance the default, and require human verification before an AI-derived claim triggers military action. Set binding design standards for companions used by minors.
Hollywood’s mistake was giving us a villain to hate. The real risk has no villain, incentives and a checking gap. The gap is closable, and closing it costs less than any of the scenarios above.
MY FORECAST: Expect the patch gap to deliver the first headline-grade failure, because its numbers are public and its targets are small utilities with no security staff. Expect the fog to follow, through a fake image believed or a real one dismissed in a crisis. The squeeze and the drought move over years, and by the time voters see the damage, reversing the choices will cost a fortune. Watch which government first reserves human-only roles by law or writes provenance into its military rules. Whoever moves first will set the template for everyone else.
If you or someone you know needs support, please get in touch with the 988 Suicide and Crisis Lifeline by calling or texting 988 in the US, or the Samaritans on 116 123 in the UK.
Related Stories
- The Backlash Against Data Centres Isn’t Anti-Tech. It’s a Bill Coming Due.
- Bill Gates Wants ‘Human-Reserved’ Jobs Protected From AI
- Meta Settles States’ Child Safety Trial for $17Bn