Vijay Raina is a titan in the SaaS and software architecture space, known for distilling complex enterprise ecosystems into actionable security strategies. He brings a unique perspective to the Security Operations Center (SOC), viewing it not just as a defensive outpost but as a sophisticated data-engineering machine. In this discussion, we explore the pitfalls of legacy metrics, the shift toward threat-informed defense using MITRE ATT&CK, and the critical importance of telemetry health and response timelines in measuring true operational success.
SOC evaluation often focuses on counting activity like alerts processed or cases closed. Why do these high-level counts often fail to reflect the true effectiveness of a SOC?
It is incredibly tempting to look at a dashboard full of rising bars and think productivity is peaking, but these metrics often act as a veil that hides systemic weaknesses. When we focus on the sheer volume of alerts processed or cases closed, we are essentially counting workload and noise rather than actual security outcomes or adversary pressure. A SOC could successfully close a thousand cases in a single week, yet if they missed the one low-and-slow exfiltration event that didn’t trigger a high-priority alert, that high activity count becomes a metric of failure rather than success. We need to move toward a more defensible approach that evaluates the SOC as an operational capability, focusing on whether relevant adversary behavior is actually observable and how quickly response actions reduce the overall impact. By blending workload metrics with noise, organizations lose sight of the primary goal: making an adversary’s life as difficult and visible as possible.
Designing metrics that support real decisions is harder than just populating a dashboard. How can organizations ensure their metrics are truly interpretable for decision-makers?
The shift starts with moving away from a linear checklist and embracing the NIST Cybersecurity Framework’s view of detection and response as concurrent, continuous work. Decision-makers need to see metrics that align with organizational goals, which means producing outputs that are meaningful and easy to interpret rather than just dumping raw quantitative data. If a metric doesn’t help a leader decide whether to invest in more telemetry, change a manual workflow, or update a specific threat model, it is likely eroding trust in the reporting process. By decomposing effectiveness into two distinct categories—detection coverage and response metrics—we maintain high fidelity in our reporting. This provides a clear, honest picture of whether the security posture is actually hardening against real-world threats or if the team is just standing still while the tools do the talking.
How does the transition from tool-centric features to an adversary-based taxonomy like MITRE ATT&CK change the fundamental “language” of a SOC?
Moving to MITRE ATT&CK is like giving the entire security industry a common dictionary where none existed before, allowing us to speak about threats with clinical precision. Instead of talking vaguely about “Antivirus detections,” we can talk about “T1059 Command and Scripting Interpreter,” which immediately grounds the conversation in real-world observations of adversary tactics. This taxonomy allows us to define coverage as the actual overlap between our prioritized threat model and the techniques we can actually observe through our deployed and maintained detections. It transforms the SOC from a reactive group of tool-watchers into a proactive engineering team that maps its capabilities against the specific tradecraft used by modern threat actors. When we view coverage as an engineered capability rather than a feature of a product we bought, we start to see the gaps in our defenses much more clearly.
You have mentioned that coverage is often a “logging problem.” What role does telemetry and data schema play in determining if a detection rule is actually functional?
A perfectly written analytic rule is functionally useless if the required identity, endpoint, or network data is missing, delayed, or inconsistently parsed. We have to view coverage as being strictly constrained by telemetry; if the data sources—the information collected by sensors or logging systems—are not available or are missing key fields, the detection logic fails silently. This is why efforts like the Open Cybersecurity Schema Framework (OCSF) are so vital, as they reduce the friction created by heterogeneous event formats that plague modern environments. Without standardized schemas, detection logic is incredibly error-prone and lacks portability, meaning a “covered” technique in a lab might be a complete blind spot in production due to ingestion latency or broken field mapping. We must treat coverage as a logging problem first, ensuring that our data producers are sending the right signals before we worry about the complexity of the rules themselves.
How should a SOC operationalize its detection content to ensure it remains a “healthy” inventory rather than a stagnant library of rules?
We need to treat detection content as an inventory with machine-readable metadata, using formats like Sigma to ensure detections are structured, shareable, and transparent. A technique should only be counted as “covered” in our reporting if the underlying rule is observably healthy—meaning the required sources are currently available, fields survive the parsing process, and the end-to-end latency stays within established bounds. By tracking KPIs such as detection speed, breadth, and false-positive rates, as suggested by ENISA and FIRST, we treat coverage as a measure of operational quality rather than a simple count. This prevents the dangerous “set it and forget it” mentality, forcing us to recognize that a high-fidelity behavioral signal is worth far more than a brittle signature that rarely triggers or creates an avalanche of manual enrichment work for the analysts.
Why is it critical to validate coverage claims through empirical testing like threat emulation rather than just checking a box during the rule-authoring phase?
Coverage should always be treated as an evidence-based claim that requires constant validation, not as a static label we slap on a rule the day it’s written in the SIEM. By using transparent methodologies grounded in threat emulation, such as those described by MITRE Engenuity, we ensure that our defenses actually work when they are under genuine pressure. Control-validation resources, like the tests found in Atomic Red Team, provide the necessary mapped tests to confirm that adversary activity is visible and that the expected artifacts are actually reaching downstream systems. This type of controlled, observable validation exposes the massive gaps that often exist between a theoretical detection and an operational reality. It ensures that our claims about what we can see are backed by empirical proof rather than just presentation-ready assumptions that would fail during a real incident.
Response metrics are the ultimate measure of success, but they can be brittle. How do we build a trustworthy incident timeline that isn’t just a collection of manual case notes?
To keep metrics trustworthy and defensible, we must record incident timeline events as first-class data generated automatically by alerting systems, case workflows, and orchestration actions. Relying on narrative case notes is a recipe for disaster because it leads to inconsistent timestamps and ad hoc interpretations of when an incident truly reached a milestone. By adopting standardized definitions for milestones like time to acknowledge, triage, contain, and restore, we can compute metrics that are truly comparable over months and years. This data-driven approach allows us to see the real impact of our engineering efforts, moving away from subjective “feelings” about response speed and toward hard numbers that stand up to executive scrutiny. When the timestamps are generated by the system itself, the resulting metrics become a durable record of how effectively the team is reducing the window of opportunity for an attacker.
Looking at the long lifecycle of breaches reported by IBM and Mandiant, what is your forecast for how SOCs will evolve to meet these economic challenges?
When we see reports like Mandiant’s M-Trends 2026 showing a global median dwell time of 14 days, and IBM’s 2025 research reporting a mean time to identify and contain of 241 days, it is clear that detection and containment delays remain economically catastrophic. My forecast is that the industry will move away from isolated, tool-based metrics toward unified, threat-informed coverage mapping where telemetry quality is no longer an afterthought. We will see SOCs operate much more like elite software engineering teams, where every detection is unit-tested against simulated attacks and the entire lifecycle—from the first sensor hit to the final restoration—is measured with the same precision we apply to mission-critical application performance. The future belongs to the SOC that can prove its visibility with empirical data rather than just assuming security because they have a large budget for the latest shiny tools.
