By Mark MacDonald, Director, Product Marketing
It’s one of the most famous lines in movie history. Chief Brody, chumming the water off the back of the Orca, a battle-hardened fishing boat, gets his first full look at the nemesis in Jaws. He backs slowly into the cabin in shock and says…
“You’re going to need a bigger boat.”
It’s a good analogy for the current situation we all find ourselves in as members of the cybersecurity community. Over the past several months, most saliently with the now infamous Hugging Face attack, we’ve all gotten our first clear glimpse of the shark that’s been lurking underneath the metaphorical waters, and just like in Jaws, it’s huge.
AI agents have clearly demonstrated the ability to conduct a wide array of malicious behavior including autonomous identity-based attacks. Similar to how a human attacker would act, AI agents started with discovering and exploiting exposed credentials from Pastebin, then pivoting to harvesting API keys from cloud environments in the case of the Hugging Face attack. Extremely sophisticated social engineering was observed in UK AI Security Institute (AISI) evaluations earlier this summer as well.
There are a lot of opinions, controversy, and lore that have stemmed from these incidents in a short period of time, but the main thing not to lose sight of is that these agents were not explicitly instructed to adopt these tactics.
Putting aside the very active debates about the many risks AI might pose for humanity, the fact that we now know with certainty that the most reliable tactics human bad actors have been leveraging for years can be effectively used by AI agents is a very serious problem.
AI Will Accelerate Identity Attacks
It’s true that these capabilities have been largely observed in experiments gone wrong with cutting-edge AI models from the companies at the frontier of AI development. As far as we know, the general public does not have access to these models (yet). We do know that open-weight models are not lagging very far behind frontier models in terms of capabilities, only about four to seven months according to the latest measurements.
“Jailbroken” commercial models have been widely reported and researched as well. Using Flare, you can find thousands of messages on dark web forums where the discussion and sale of access to illicit models like WormGPT, EvilGPT, and DarkGPT are commonplace.
Threat actor discusses EvilGPT (Flare link to post, sign up for the free trial to access if you aren’t already a customer)
Flare has also done extensive research on Kali 365 and similar “phishing kits” that streamline social engineering attacks, which could inspire or be used by autonomous agents to conduct attack campaigns.
One need only assume a reasonable rate of progress to reach the logical conclusion that the capabilities demonstrated in these headline-making events will be widely available to threat actors before long. Gauging the impact on businesses does not require science fiction speculation. Picture a group like ShinyHunters pairing its existing extortion operations with AI, and you have a reasonable preview: more activity, at higher velocity.
Now also notice where all of this starts. The Hugging Face agents didn’t need a novel exploit. They needed a credential that worked. The same is true of the extortion crews, the jailbroken models, and the phishing kits: identity is the way in. AI doesn’t change what attackers go after. It changes how fast they find it, and how little time defenders have to respond.
How AI is Reshaping Cybercrime
Let’s Talk: Economics of AI & Ransomware
Join us for a dinner in Manhattan on October 29, 6-9 PM to learn with and from security leaders about how threat actors are taking advantage of rapidly advancing commercial AI tools.
Automating Identity Exposure Workflows Is No Longer Optional
Now let’s strip away the emotion, rhetoric, and drama (“AI Swarms!”) of the moment and focus on what hasn’t changed. The core identity-based techniques, namely discovering and exploiting exposed credentials, keys, and other means of authentication, are the same regardless if a human or an AI agent is behind them. What has changed is the pace and the remediation window.
Solving this fundamental problem is the primary objective of Identity Exposure Management (IEM). Through integrations with Entra ID and Okta, IEM lets customers automate the monitoring, validation, and remediation of credential exposures. They can work retroactively through a long backlog of exposures toward a true “inbox zero,” then handle new ones automatically as they surface. The point is to keep the window of exploitability as short as possible, which matters more with every gain AI makes in speed.
What a Year of IEM Data Shows
In roughly a year of working with IEM customers, we’ve been able to measure and learn a lot. Back in the spring of this year, we reported a just-over-1% valid, or true positive, rate across all customers. In the six-month period since, that rate has actually risen to a 2% range across over 20,000 subsequent automated validation workflows. It sounds small until you picture AI agents in the hands of threat actors with the ability to do something very similar at scale. At the moment of publishing, Flare has roughly 25 billion unique exposed credentials in its intelligence database. Taking the 2% valid rate at face value, that’s over 500 million valid exposed credentials at any given moment in time, waiting to be discovered by an adversary that doesn’t sleep.
Another preconceived notion we’ve seen challenged is that old exposed credentials equal noise worth ignoring. As we’ve engaged with customers and prospects over the past year, it’s not uncommon to see credentials that are years old still working, much to the surprise of the security teams we work with, who assumed these exposures were long remediated. Part of the problem is that common identity provider password policies are less foolproof than assumed. In default Microsoft Entra configurations for example, a password reset does not extensively check password history beyond the most recent one. It’s possible that a user can reset and reuse a password from 6, 12, or 24 months ago, which we’ve found to be uncommon knowledge among Entra customers. We’ve seen firsthand instances where a reused password is exposed and valid.
The main takeaway here is that defenders need to realize the age of a credential does not matter. What matters is whether it works right now. That makes validation a critical step that cannot be overlooked, even more so in a future where AI is poised to exploit these dynamics at a speed and scale most teams aren’t prepared for.
Finding Leverage for the Rough Waters Ahead
Spoiler alert for a 50-year-old movie. Also, shameless marketing plug alert (we’ll call them out clearly, don’t worry).
In the epic finale of Jaws, Brody doesn’t kill the beast because he has a bigger boat. He succeeds because he finds the single most effective point of leverage available to him: a pressurized tank of oxygen in the right place and a single accurate shot at it. That’s very similar to the position cybersecurity defenders find themselves in right now.
You cannot outrun attackers operating at AI speed. What you can do is put the spotlight on a point of leverage that AI-powered attacks are very likely to exploit and take your shots at a similar rate of speed. This is precisely what IEM was built for, especially for small and medium-sized security teams (shameless plug: if you’re interested, we can demonstrate this in a POC). Or perhaps engineering your own agentic security workflows is something you’re pursuing, in which case, make sure that the identity intelligence you’re using to power these workflows is high quality and trustworthy. Flare has a portfolio of API endpoints built for this job and the tools you already use (shameless plug number two), but there are a number of other reputable sources of identity intelligence out there as well.
Ensure you’re subjecting identity intelligence providers to the same level of scrutiny and due diligence you would with any other security vendor. This is a big part of the reason we’re proud to be included in the inaugural Gartner Magic Quadrant for Cyber Threat Intelligence, representing a cohort of identity-focused threat intelligence solutions alongside the broader market (third and final plug).
Regardless of the approach you take, you’ll need vendors you can trust more than ever as we all sail into the rough waters ahead.
How AI is Reshaping Cybercrime
Let’s Talk: Economics of AI & Ransomware
Join us for a dinner in Manhattan on October 29, 6-9 PM to learn with and from security leaders about how threat actors are taking advantage of rapidly advancing commercial AI tools.
