Why Defenders Have the AI Advantage (And How to Keep It)
Splunk's Ryan Fetterman reveals how security teams can leverage LLMs to outpace attackers—and introduces The Threat Hunter's Cookbook
While much of the cybersecurity industry frets about AI empowering attackers, Ryan Fetterman has a contrarian view: defenders currently hold the advantage in the AI arms race. As Senior Security Strategist at Splunk SURGe and co-author of the newly released "The Threat Hunter's Cookbook," Fetterman believes the nature of security operations work plays directly to AI's strengths.
"I think the defender has the advantage because so much of the work that we do in the SOC is based around text-based data, understanding it, generating it," Fetterman explained during our interview at Black Hat 2025. "Those are the core strengths of models, and that's the core of a lot of the processes that flow through the SOC: threat intelligence, consuming it, writing it, looking at alerts, understanding machine data, code interpretability."
This perspective, which Fetterman acknowledges might be "contrary to the median opinion," stems from practical experience deploying AI in real security operations rather than theoretical concerns about AI-powered attacks.
From Theory to Practice: Model-in-the-Loop Threat Hunting
Fetterman's optimism isn't just theoretical, it's backed by concrete results from Splunk's research into what he calls "model-in-the-loop threat hunting." The approach addresses one of the most persistent challenges in security operations: the overwhelming volume of data that requires human analysis.
Take PowerShell script analysis, a classic example of the "living off the land" problem that plagues security teams. Both legitimate administrators and malicious actors use PowerShell extensively, creating a needle-in-haystack scenario where human analysts must manually classify thousands of scripts to identify genuine threats.
"Typically, you need a human in the loop to look at all these things and make a determination of whether something is legitimate or malicious functionality," Fetterman said. "What we did was we wrote a query that would pull a bunch of those events and then, rather than having an analyst go through 2,000 different scripts and classify them, we fed all of these scripts into open-weight LLMs."
The results were striking. Using off-the-shelf models like Llama 3, the team achieved around 80% accuracy with nearly perfect recall; meaning they caught almost all true threats while generating manageable false positives. More importantly, the speed improvement was dramatic: from five minutes per event for human analysis to two seconds per event for AI analysis, a 150x speed improvement.
"That ultimately was the problem we wanted to solve," Fetterman noted. "Everyone agrees that AI for attackers is increasing efficiency and productivity. So if we're looking for opportunities to go faster on defense, to keep that parity, this is the kind of case we're looking to solve."
The Cookbook Approach to Threat Hunting
This practical experience with AI implementation informs Splunk SURGe's broader approach to threat hunting, codified in their new "Threat Hunter's Cookbook." Released at Black Hat 2025, the guide represents a systematic approach to the "black box" of analysis that often mystifies security teams.
"With so many ways to kick off a hunt, it can be hard to select the best approach," the team notes. "Even if you have tried-and-true ways for hunting, this can ultimately cap the ceiling of your potential results."
The cookbook organizes threat hunting methods—searching and filtering, sorting and stacking, clustering, grouping—into practical "recipes" that follow a problem-solution-discussion format. Built on the PEAK Threat Hunting Framework (winner of the 2024 SANS Difference Makers award for Innovation), it provides both methodology and practical SPL queries for Splunk users.
Navigating the Complexity Trade-off
Despite his enthusiasm for AI in security, Fetterman emphasizes the importance of matching solution complexity to problem complexity. "Sometimes machine learning is needlessly complex," he said. "I tend to try to emphasize that you don't want to be more complex than you need to be. That's a battle you fight a lot in security, especially when customers are eager to say they're using machine learning or integrating AI products."
This philosophy extends to how organizations should think about AI-powered threats. Rather than panicking about theoretical AI attacks, Fetterman suggests a more measured approach: "We should prepare to face them in ways that, I like the idea of, purple teaming with AI agents. You're expecting adversaries to build red agents to deploy against you. You should build them yourself and start testing them against your own defenses."
The Future of AI-Powered Security
Looking ahead, Fetterman sees the future not in AI replacing human analysts, but in changing the ratio of humans to automated agents. "We may have more of a ratio of analysts to agents, where an analyst is overseeing different agents that are focused on different tasks," he explained. "That seems like a much better, more scalable, more controllable future for how we integrate AI."
This vision aligns with Splunk's broader strategy of creating domain-specific models for security use cases. The company is already exploring advanced applications, including behavioral fingerprinting that uses LLMs to analyze post-login user behavior and identify suspicious activity patterns.
Early Warning: AI Malware in the Wild
Fetterman's research has also uncovered early examples of AI-powered malware, providing a glimpse into how attackers might leverage these technologies. A recent report from CERT in Ukraine, attributed to Russian government actors (APT 28 or 29), described Python-based malware that used the Hugging Face API and prompts to the Qwen Coder model instead of containing traditional hardcoded instructions.
"Instead of containing instructions, it actually used prompts to a model and was asking the model to return commands to run on the system," Fetterman explained. "It's a totally different paradigm of how malware works, maybe a new category of how attackers are trying things."
While the advantages of this approach remain unclear (the malware was detected), it demonstrates that both sides are actively experimenting with AI integration.
Practical Recommendations for Development Teams
For developers building security tooling, Fetterman offers several key recommendations:
Start with domain-specific models: Generic AI models can be effective, but security-specific models trained on relevant data will perform better for specialized tasks.
Implement human-in-the-loop design: Rather than pursuing full automation, design systems where AI handles initial triage and analysis while humans make final decisions.
Prepare for agent-based threats: As attackers develop AI agents for reconnaissance, initial access, and discovery, build purple team capabilities to test your defenses against similar tools.
Don't overextend: While organizations "can't afford to not be doing AI stuff right now," be cautious about bleeding-edge implementations in rapidly evolving areas like multi-agent frameworks.
One often-overlooked advantage of platforms like Splunk in the AI era is their potential to reduce security tool sprawl. With the average Fortune 500 company managing 70+ security solutions, the ability to build custom AI-powered analytics and dashboards within a unified platform becomes increasingly valuable.
"Because of the extensibility and customization of Splunk, you are free to build more things yourself," Fetterman noted. "That's a key way that I've seen products not be needed—if you have Splunk, then you can build custom dashboards, integrate with APIs, and eliminate the need for certain gaps or certain categories of problems."
Staying Ahead of the Curve
As AI continues to reshape cybersecurity, Fetterman's message is clear: the tools and techniques exist today for defenders to maintain their advantage, but success requires thoughtful implementation rather than wholesale adoption of the latest AI trends.
The release of "The Threat Hunter's Cookbook" represents more than just documentation, it's a systematic approach to leveraging both human expertise and AI capabilities in pursuit of more effective threat detection. For organizations wondering how to navigate the AI revolution in cybersecurity, Fetterman's advice is practical: start with clear problems, use appropriate tools, and always keep humans in the loop.
As the cybersecurity landscape continues to evolve, the organizations that thrive will be those that, like Splunk SURGe, combine rigorous methodology with practical AI implementation—turning the promise of artificial intelligence into measurable improvements in security outcomes.
