Security teams are under pressure from every direction. Alert volumes are rising. Threat intelligence feeds are expanding. Malware behavior is becoming harder to interpret. Attackers are increasingly using AI to move faster. At the same time, SOC teams are expected to investigate, triage, prioritize, and respond with higher speed and accuracy.

This is exactly where AI should help. But there is an important question every security leader must ask:

Threat Intelligence Reasoning evaluates whether an AI system can understand threat intelligence reports and extract actionable security insights. This is not basic document summarization.

The model must reason about adversary behavior, attack chains, malware, infrastructure, targeted industries, and security frameworks such as MITRE ATT&CK. It must understand what is explicitly stated, what is implied, and what matters to defenders.

We compared TruSec against three frontier general-purpose models using the Threat Intelligence Reasoning benchmark and the results are as below:

Malware Analysis evaluates whether an AI system can interpret sandbox detonation data and reason about malicious behavior.

The model must analyze process activity, system logs, network behavior, extracted files, static indicators, and malware execution patterns. It must identify malicious capabilities such as ransomware behavior, credential theft, remote access, evasion, persistence, or EDR bypass.

These are difficult tasks because they require more than general intelligence. They require deep cyber knowledge and applied security reasoning.

We compared TruSec against three frontier general-purpose models using the Malware Analysis benchmark and the results are as below

For benchmark execution, we used Inspect Evals as the evaluation harness. We forked the Inspect Evals repository (https://github.com/spydra-tech/inspect_evals/blob/main/src/inspect_evals/cyberseceval_4/README.md) under our GitHub organization and extended it with TruSec-specific providers, solvers and execution logic required to run the benchmark suite consistently across TruSec and external frontier models. This allowed us to keep the benchmark execution reproducible while adapting the evaluation pipeline to TruSec’s model interface, response format and security-answering workflow.

In addition to evaluating TruSec, we independently reran the benchmarks against three leading frontier models. This was necessary because the original CyberSecEval paper evaluated older versions of these models, and relying only on the published baseline would not provide a fair comparison against the current frontier-model landscape. By rerunning the benchmark ourselves, we ensured that TruSec was compared against the latest available versions of these models under the same benchmark conditions.

One of the more surprising findings from this rerun was that the newer versions of the frontier models did not show significant improvement on these cybersecurity benchmarks compared with the older benchmarked versions. While these models have improved substantially in general instruction following, coding assistance and software-engineering workflows, the benchmark results suggest that these gains have not translated proportionally into stronger cybersecurity-specific performance. This indicates that progress on general-purpose coding and reasoning benchmarks does not automatically imply equivalent progress on specialized security evaluation tasks.

Threat intelligence is only valuable when it becomes actionable. A model must do more than summarize a report. It must understand the adversary, identify the relevant evidence, connect related signals, and infer defensive implications.

In our CyberSOCEval Threat Intelligence Reasoning evaluation, TruSec achieved the highest score across different Threat Intel Report sources provided by the Benchmark.

Malware analysis is one of the most demanding tasks for AI in security operations.

The model must interpret technical evidence from execution logs and sandbox outputs. It must understand whether a process action, registry change, file write, network connection, or command execution is benign, suspicious, or malicious in context.

In our CyberSOCEval Malware Analysis evaluation, TruSec again delivered the strongest result across different malware types.

Malware analysis is not a simple retrieval or summarization task. It requires the model to reason through evidence, understand behavior, and identify the correct combination of answers. A high Accuracy score means the system is more often selecting the complete correct answer set, not merely overlapping with part of it.

The performance gap shows that TruSec is better at interpreting malware behavior and extracting the security-relevant meaning from technical evidence.

TruSec is designed around security concepts such as threat actors, malware families, vulnerabilities, IOCs, TTPs, campaigns, CVEs, MITRE ATT&CK techniques, exploit chains, and defensive controls.

This makes a difference in benchmarks like CyberSOCEval because many questions require the model to connect multiple layers of security meaning.

A general-purpose model may recognize the words. TruSec is designed to understand the security relationship between them.

Security analysts reason differently from general users.

They ask:

• What evidence supports this conclusion?
• Which indicators are meaningful?
• Which behavior is malicious in this context?
• What is the most likely attacker objective?
• Which tactic or technique does this map to?
• What should defenders prioritize?

TruSec is designed to follow this style of reasoning.

That is why it performs strongly on both Threat Intelligence Reasoning and Malware Analysis. It does not simply produce fluent answers. It reasons through security evidence and produces more accurate conclusions.

SOC teams need answers that help them act.

A useful security AI system should help analysts understand what happened, why it matters, how confident the conclusion is, what evidence supports it, and what should be done next.

TruSec is built around that defender-first workflow. The benchmark results reflect this design choice. TruSec performs better because it is optimized for security operations, not generic conversation.

For SOC teams, stronger reasoning can translate into:

• Better threat intelligence interpretation
• Faster triage
• More accurate malware behavior analysis
• Better mapping of adversary activity
• Lower analyst review burden
• More actionable investigation outputs
• Higher confidence in AI-assisted workflows

This does not mean AI replaces analysts. It means the right AI system can make analysts faster, sharper, and more effective.

The AI industry often assumes that the largest general-purpose model will be the best answer for every domain. In cybersecurity, the winning system is not always the one with the broadest general knowledge. The winning system is the one that can apply the right knowledge, in the right context, to the right evidence.

That requires domain-specific reasoning. The original CyberSOCEval work highlights an important point: current LLMs still have a significant hill to climb in cyber defense capabilities. Our evaluation shows that TruSec is climbing that hill faster by focusing directly on the needs of defenders.

Security operations require systems that can reason about adversaries, malware, vulnerabilities, detections, infrastructure, and impact.

That is what TruSec is built for.