For months, the heads of frontier AI labs have been warning us that incidents like this summer’s Hugging Face hack could represent “early warning shots” before more menacing attacks in the years to come. Equipped with their own doomsday bunkers, some CEOs have begun instructing their teams to run preparedness scenarios for the day after a “catastrophic AI incident.”
“We are willing to accept some things going wrong,” OpenAI CEO Sam Altman told Vanity Fair’s global editorial director, Mark Guiducci, last month, adding that “there is a coming tidal wave of cybersecurity problems that are going to come from these open-source models.”
Just in time for spooky season, those warning shots got a little louder. On Wednesday, October 21, two serious security vulnerabilities were revealed in PHP, the programming language behind more than 70% of the internet’s web pages. Some cybersecurity professionals have already begun to call this “Black October,” thanks to the unprecedented spate of hacks across the globe.
AI did play a role in all this—but perhaps not the one you’d expect. Security researchers had actually used tools developed by OpenAI to discover the bugs months ago—and had been rushing to patch them up before they got into the hands of bad actors. Unfortunately, those vulnerabilities leaked last week, sending governments and businesses scrambling to protect their websites before hackers—perhaps armed with some of the very same AI tools as those defending against them—moved in.
Yet even the frontier labs’ ability to keep pace was called into question on Monday morning, when The Washington Post published a story about how Cathedral, a new cybersecurity company founded by four ex-DOGE staffers, successfully infiltrated some of Anthropic’s internal systems in just 12 hours. The article dropped just hours before Anthropic filed pre-IPO documentation detailing the potential risks to its own revenues and the world at large if its cybersecurity game were found to be lacking.
To the cybersecurity community, this was just the latest in a series of startling escalations. Joshua Achiam, who resigned from his post as OpenAI’s chief futurist this summer after nine years at the company, wrote on X: “Feels like March 1, 2020 for Cyber. Not normal, something is happening. Internet lockdowns in a couple of months?”
While these anxieties may not have reached your average American yet, business leaders have been growing increasingly concerned. In mid-September, executives from some of the world’s largest companies gathered for a special meeting of the Partnership for New York City. Their special guests that day were Anthropic whistleblower Jacob Coxon and former Department of Defense official Mark Beall, who had come to warn them about the threats posed by emerging AI capabilities.
The co-chairs, Pfizer CEO Albert Bourla and Tishman Speyer CEO Rob Speyer, looked out at their screens full of anxious CEOs from hedge funds and banks, as well as insurance, real estate, and pharmaceutical companies, and noted that it was one of their largest gatherings in years. One big topic on the minds of those gathered: AI-enabled cyberattacks.
During the Q&A portion of the talk, a representative from Coinbase raised his hand to ask what companies like his could do to combat the threat of “open-weight models in the hands of bad actors,” from cybercriminals to nation-states. (Coinbase CEO Brian Armstrong, meanwhile, has been one of the more outspoken advocates of deregulated, open-source AI.)
Earlier this month, Japan declared a “state of emergency” following sweeping cyberattacks on more than 20 major companies, from financial firms to rail operators. French authorities are still piecing together the mystery of how and why hackers targeted supermarket refrigerators around the country, spoiling millions of euros’ worth of food. Meanwhile, a new report from CrowdStrike suggests that recent attacks on South Korea’s financial sector were allegedly carried out by a single 26-year-old in China using an open-source AI tool.
Even here, cyberattacks have become so commonplace that you may have missed them in the flurry of headlines. (Did you catch this month’s massive breach of FBI employees’ personal details? Iran’s hit on more than 30 water systems in Minnesota this summer? And that’s to say nothing of mounting efforts by Russia to influence the midterm elections.)
“Feels like March 1, 2020 for Cyber.”
I was surprised to learn that the concerns of most cybersecurity experts are both less apocalyptic and more present than the kinds of visions filling the heads of AI CEOs. In short, they believe that AI systems haven’t yet caused a paradigm shift in terms of the scale of damage they can inflict. What increased access to AI tools has done is lower the barrier to entry for even unskilled hackers to target low-hanging fruit.
“Over the coming weeks and months, the more likely scenario is a local hospital network in Ohio losing power and patients on life support dying as a result; a utility network in Texas taken offline; the lights going out across half of Wisconsin,” says Hamza Chaudhry, the AI and national security lead at the Future of Life Institute.
“We’re just in an exceptionally unresilient moment, coming up against systems which are designed to exploit that lack of resilience,” he tells me. “The largest providers,” however, “also have the most resources to defend themselves, including early access to frontier AI models for defense.”
He concedes that there is one more scenario that is keeping him up at night. “I’m very concerned about the so-called geopolitical Hugging Face moment.”
At the Munich Security Conference earlier this year, his group ran a simulation for an audience of national security officials from across the West: Imagine China develops a Mythos-level cyber-capability, which then gets into the hands of the mercenaries at the Wagner Group, who deploy it in Mali to target French military infrastructure. Then, much like in the Hugging Face incident, the agents go on an unintended side quest and attack mainland military infrastructure inside Europe. And that’s when things spiral out of control. “There was a real chance that Article 5 would be invoked and NATO would declare war on Russia” in this simulation, Chaudhry tells me.
A high-level executive at one of the world’s largest energy suppliers tells me he’s been losing sleep over aging infrastructure, budget cuts, and staffing shortages. Small utilities, he says, are plagued by “patches that might be easy to fix given the resources necessary.” But for now, “operators are running on Windows NT,” he says, referring to Microsoft’s 1990s operating system. New infrastructure coming online to support data center buildouts isn’t much better off. This infrastructure is “being built at such a speed [that] there’s no regulation and there are no hard standards,” says the energy executive. “Security is an afterthought, and that scares me a whole lot.”
This is the part of the story where I’d love to reassure you that the powers that be have been hard at work bolstering our defensive systems. The reality is, the Cybersecurity and Infrastructure Security Agency has lost nearly a third of its workforce and hundreds of millions of dollars in funding, and it has failed to appoint an official director to lead the agency since the Trump administration took office in January of last year. But things at the state and local levels are what really worry the experts I spoke with. A recent Deloitte survey of state chief information security officers (CISOs) found that their budgets are failing to keep pace with growing threats—and have even faced cuts in 16% of the country.
“For some reason, amidst increasing cyber-capabilities, our defenders are running out of money,” Soham Mehta, a research fellow at the Foundation for American Innovation, tells me. “Great.”
“Security is an afterthought, and that scares me a whole lot.”
There is at least some indication that authorities are beginning to wake up. Earlier this month, the new White House AI task force met for the first time to discuss cybersecurity and critical infrastructure protection, among other topics. Many are hopeful that frontier AI will do more to help companies and governments defend themselves than it will to help their attackers—as in the case of OpenAI’s assistance in discovering the PHP flaws. In a survey of more than a hundred national security officials last month, 81% said that defensive capabilities are very likely to improve with the help of AI.
None of the experts I spoke with advised me to load up the bug-out bag just yet. For now, our fate is largely in the hands of the government officials and executives who have to make the choice to invest in updated security systems.
They largely gave the same advice I’ve been working off for years. Set up a password manager, as well as two-factor authentication wherever it’s available. Beware of any unusual messages, and don’t drop your credit card details into any suspicious websites. And perhaps most importantly: Contact your local representatives to nudge them on updating that 1990s software.
-
Erika Kirk Doesn’t Really Want to Talk Politics. So How’s She Going to Lead Turning Point USA?
-
A Deep Dive Into How Larry Ellison Built His Influence
-
Making the Elizabeth Holmes Doc Fried Nathan Fielder’s and Lance Oppenheim’s Brains
-
“History Will Judge”: Fox News Anchor Bret Baier on Donald Trump and Pete Hegseth’s War With the Press
-
Inside Lane Kiffin’s Trash-Talking, “Joey Freshwater” Era With Nick Saban at Alabama
-
From the Archive: Inside Elizabeth Holmes’s Chilling Final Months at Theranos
