A useful security observation
Detection quality is the degree to which a detection finds the defined harmful behavior soon enough and with enough context for a useful response. It includes visibility, logic, data quality, coverage, false positives, false negatives, delay, robustness, ownership, and containment outcome.
Define the hypothesis
State adversary or failure behavior, protected asset, required events, expected benign cases, analytic logic, threshold, time window, affected identity and resource, and response. Name the exact procedure observed, not only a broad ATT&CK technique.
Map collection points and blind spots. A route log can see a request and not endpoint malware or a direct origin.
Test and measure
Use unit data, historical replay, controlled simulation, failure injection, and production validation. Measure source availability, field completeness, time skew, end-to-end latency, precision, recall where ground truth exists, analyst time, disposition, and time to containment.
Track coverage by threat and asset, not by number of rules. Retire or redesign rules that no longer support a useful action.
Failure and residual risk
Perfect test fixtures can hide parser, deployment, and source failures. High precision can miss new variants. High volume can exhaust analysts. Technique count can overstate coverage. A blocked test can produce evidence different from a real attacker.
Detection does not prevent the first action and cannot reverse completed harm. Recovery and preventive control still matter.
Pomerium boundary
Pomerium logs can support detections for route decisions and covered access. Operators must monitor log delivery and schema and correlate other systems. A detection built only on Pomerium has no coverage for bypass traffic, endpoint activity, identity-provider administration, or final application action unless those sources join it.
Evaluation checklist
- Which exact behavior, asset, identity, source fields, time window, and response define the hypothesis?
- Does the deployed source observe every required step with current, complete, ordered data?
- What positive, benign, variant, missing-source, delayed-source, and bypass tests ran?
- Are analyst cost, false closure, containment time, and blind spots measured with rule matches?
- Does coverage describe real assets and procedures rather than rule or framework counts?
