CrowdStrike & Meta Launch Benchmarks for AI in Cybersecurity
New benchmarks outline how LLMs needs to be examined in the SOC – measuring actual threats, workflows, and outcomes to assist defenders
Fal.Con 2025, Las Vegas–CrowdStrike (NASDAQ: CRWD) at present, in partnership with Meta, launched a brand new suite of benchmarks – CyberSOCEval – for evaluating how AI techniques carry out in real-world safety operations. Built on Meta’s CyberSecEval framework and CrowdStrike’s main menace intelligence and cybersecurity AI knowledge experience, this suite of open supply benchmarks helps set up a brand new framework for testing, choosing, and leveraging massive language fashions (LLMs) in the safety operations middle (SOC).
Cyber defenders face an awesome problem from the inflow of safety alerts and evolving threats. To outpace adversaries, organizations should embrace the most recent AI applied sciences. Many safety groups are nonetheless early in their AI journeys, significantly in utilizing LLMs to automate duties and drive effectivity in safety operations. Without clear benchmarks, it’s troublesome to know which techniques, use instances, and efficiency requirements ship a real AI benefit in opposition to real-world assaults.
Meta and CrowdStrike are addressing this problem by introducing CyberSOCEval, a collection of benchmarks that assist outline what efficient AI seems like for cyber protection. Built on Meta’s open supply CyberSecEval framework and CrowdStrike’s frontline menace intelligence, CyberSOCEval evaluates LLMs throughout important safety workflows resembling incident response, malware evaluation, and menace evaluation comprehension. By testing AI techniques’ means in opposition to a mix of real-world adversary tradecraft and expert-designed safety reasoning eventualities primarily based on noticed adversarial techniques, organizations can validate efficiency beneath stress and show operational readiness. With these benchmarks, safety groups can pinpoint the place AI delivers most worth, whereas mannequin builders acquire a North Star for enhancing capabilities that improve ROI and SOC effectiveness.
“At Meta, we’re dedicated to advancing and maximizing the advantages of open supply AI – particularly as massive language fashions turn into highly effective instruments for organizations of all sizes,” stated Vincent Gonguet, Director of Product, GenAI at Superintelligence Labs at Meta. “Our collaboration with CrowdStrike introduces a brand new open supply benchmark suite to judge the capabilities of LLMs in actual world safety eventualities. With these benchmarks in place, and open for the safety and AI neighborhood to additional enhance, we will extra rapidly work as an business to unlock the potential of AI in defending in opposition to superior assaults, together with AI-based threats.”
“When two leaders like CrowdStrike and Meta come collectively, it’s bigger than collaboration, it’s about setting the route of cybersecurity for the AI period,” stated Daniel Bernard, chief enterprise officer at CrowdStrike. “By combining CrowdStrike’s adversary intelligence and management in AI-native cybersecurity, with Meta’s AI analysis experience and huge dataset, we’re serving to clients – and cybersecurity as a sector – undertake AI techniques with confidence. This partnership units a brand new bar for how AI in the SOC needs to be constructed and deployed, empowering defenders to remain forward of the adversary.”
The CyberSOCEval open supply benchmark suite is now obtainable for the AI and safety neighborhood to make use of to judge mannequin capabilities. To entry the benchmarks, please go to Meta’s CyberSecEval framework.
The submit CrowdStrike & Meta Launch Benchmarks for AI in Cybersecurity first appeared on AI-Tech Park.