Detection Engineering: Writing SIEM Use Cases That Survive Tuning
Most SIEM use cases are tuned into silence within a year. How to write detections against attacker behaviour, document them with the ADS framework, validate them against a real attack, tune without going blind, and measure coverage honestly.
Security Operations
Most SIEM use cases die the same way. They go live with vendor defaults, fire too often, get tuned by suppression until they are quiet, and are never checked again. Detection engineering treats each detection as a small piece of software: written for a stated purpose, documented, tested against a real attack, owned by someone, and measured. Here is how to build detections that stay useful after the first round of tuning.
In short. Write every detection against a specific attacker behaviour, not a specific tool. Document its goal, its blind spots, its expected false positives and how to prove it works before it goes live. Tune with narrow, documented exceptions rather than broad suppression. Re-test detections on a schedule, because the data they depend on changes underneath them. And feed every penetration test and red team finding back in as a detection opportunity.
Why use cases decay
A SIEM fills up with rules faster than anyone can maintain them. Each was reasonable when it was written. A year later many of them are silently broken or silently ignored, for reasons that have nothing to do with the attacker.
| How a detection decays | What it looks like |
|---|---|
| Written for nobody in particular | A vendor default enabled because it was in the content pack, with no statement of what attack it is meant to catch |
| Tuned into silence | Noisy hosts, users or whole subnets excluded until the rule stops firing, including for the attack it was written for |
| Broken by a data change | A log source renamed a field, an agent was upgraded or a collector stopped forwarding. The rule still runs and returns nothing, which looks the same as "no attack" |
| Too specific to survive | Matches a file hash or an IP address from one report, which the attacker changes for the next campaign |
| Owned by nobody | The engineer who wrote it has moved on, and nobody knows whether it is safe to change |
None of these produce an error. A detection that has quietly stopped working is indistinguishable from a quiet week, which is why the discipline matters.
Detect behaviour, not artefacts
David Bianco's Pyramid of Pain is the most useful single idea in detection design. At the bottom are indicators that cost an attacker almost nothing to change: file hashes, IP addresses, domain names. At the top are their tactics, techniques and procedures, which are expensive to change because they reflect how the attacker actually operates. A detection for a hash catches one sample. A detection for a technique catches every tool that uses it.
Indicator matching still has a place, usually fed automatically from threat intelligence. But the detections your engineers write by hand should target behaviour, and MITRE ATT&CK is the common language for describing it. Mapping each detection to an ATT&CK technique also tells you where your coverage is thin.
What a detection is made of
Palantir's Alerting and Detection Strategy framework, published openly in 2017, is the best-known way to document a detection. It exists, in its authors' words, because "there was a lack of rigor around the creation, development, and implementation of an alert, which led to sub-optimal alerts going to production without documentation or peer-review". Every detection is written up in nine sections before it can go live. Here they are, filled in for one detection: Kerberoasting, the service-ticket cracking technique an internal penetration test often uses.
| ADS section | Worked example: Kerberoasting |
|---|---|
| Goal | Detect a user requesting Kerberos service tickets in a way that suggests offline password cracking of service accounts. |
| Categorization | MITRE ATT&CK T1558.003, Credential Access. |
| Strategy abstract | Watch domain controller service-ticket events for tickets requested with weak RC4 encryption, and for one account requesting tickets for many different services in a short time. |
| Technical context | Domain controllers record service-ticket requests in Windows event 4769, including the encryption type; RC4 appears as 0x17. Requires that audit policy is enabled on every domain controller and the events reach the SIEM. |
| Blind spots and assumptions | An attacker who requests AES tickets for accounts that support AES avoids the encryption-type signal, so the volume-based condition must stand on its own. A domain controller with auditing disabled is invisible. |
| False positives | Legacy applications that still negotiate RC4; vulnerability scanners and monitoring tools that touch many services. List them by name, not by subnet. |
| Validation | In a test environment, request RC4 service tickets for several test service accounts from an ordinary account, and confirm the alert fires with the right user and host. |
| Priority | High when the requesting account is not a known service or tool, because successful cracking often leads to privileged access. |
| Response | Identify the requesting host and user, check for other credential-access activity, and rotate passwords for the service accounts whose tickets were requested. |
The two sections teams most often skip are the most valuable. Blind spots tell the next engineer what this detection does not cover, so the gap is visible rather than assumed away. Validation turns "we have a rule for that" into "we have proven the rule fires".
Detection as code: the lifecycle
Treating detections as code means they live in version control, change through review, and are tested before deployment, the same as application code. Writing them in Sigma, an open signature format that its maintainers describe as "for log files what Snort is for network traffic and YARA is for files", keeps them portable: one rule can be converted to queries for different SIEM platforms.
Step 1 — Start from a threat, not a log source
Pick the behaviour to detect from threat intelligence relevant to your sector, from incidents, or from the attack paths your own penetration tests and red team exercises found.
Step 2 — Confirm the data exists
Check that the events the detection needs are collected, from every relevant system, with the fields populated. If they are not, the first piece of work is a logging change, not a rule.
Step 3 — Write it and document it
Write the logic and fill in all nine ADS sections. If a section cannot be completed, the detection is not ready.
Step 4 — Test both ways
Run the attack in a test environment to prove a true positive. Run the rule against a recent window of production data to estimate how often it will fire on normal activity.
Step 5 — Review and deploy with an owner
A second engineer reviews logic and documentation. The detection goes live with a named owner, a priority and a response runbook the analysts can follow.
Step 6 — Measure, re-test and retire
Track how often it fires and how often it is right. Re-run the validation on a schedule and after any change to the data source. Retire detections that no longer earn their place.
Tuning without going blind
Tuning is necessary. The question is whether each change removes noise or removes coverage. The difference is usually how narrow the exception is, and whether anyone wrote down why it exists.
| Tuning that blinds | Tuning that keeps the detection |
|---|---|
| Exclude a whole host, subnet or user group | Exclude the specific process, path and account combination that causes the noise |
| Raise a threshold until the rule stops firing | Add context, such as whether the account is privileged or the host is a server, and route low-context matches to a lower priority |
| Disable a rule that is "always noisy" | Convert it to a scheduled hunting query that an analyst reviews, while the noisy source is fixed |
| Exceptions added directly in the console with no record | Exceptions in version control with a reason, an owner and a review date |
Measuring a detection programme
A count of rules says nothing about protection. These measures say more, and none of them need a target set on day one: baseline them for a quarter first.
- Validated coverage. Of the ATT&CK techniques you have decided matter most to you, how many have a detection whose validation has been run successfully in the last quarter. A technique only counts as covered if the test passed.
- Precision per rule. The share of a rule's alerts that analysts confirm as worth investigating. A rule that is almost never right is costing triage time and training analysts to ignore it.
- Silent rules. Detections that have not fired at all in a long period. Some are fine; some are broken. Re-validate them to find out which.
- Data source health. Whether every source each detection depends on is still sending complete data.
- Findings converted. How many attack steps from your last penetration test or red team exercise now have a validated detection.
For Indian organisations, there is a logging floor under all of this. CERT-In's directions of April 2022 require ICT system logs to be kept securely for a rolling 180 days within Indian jurisdiction. Detections only work on what you collect, and investigations only work on what you keep.
Where offensive testing fits
The best source of detections is the attack paths that actually worked against you. Every step in an internal penetration test report is a candidate: name-resolution poisoning, Kerberoasting, certificate abuse, lateral movement. A purple team exercise goes one step further, running each technique while the detection engineers watch, so a gap becomes a validated rule in the same session. Our purple teaming guide covers how those sessions run.
Frequently Asked Questions
Click any question to expand the answer.
QWhat is detection engineering?
The practice of designing, documenting, testing and maintaining the rules that turn security logs into alerts, treating each detection like a small piece of software with a stated purpose, an owner and a test that proves it works.
QWhat is a SIEM use case?
A defined scenario the SIEM should detect, such as credential cracking or data exfiltration, together with the rule logic, the data it needs and the response it triggers. Detection engineering is the discipline of building and maintaining use cases so they stay accurate.
QWhat is the Alerting and Detection Strategy framework?
A documentation framework published by Palantir's incident response team in 2017. Each detection is described in nine sections, namely goal, categorization, strategy abstract, technical context, blind spots and assumptions, false positives, validation, priority and response, before it can go into production.
QWhat is Sigma?
An open, generic signature format for describing log events, which its maintainers describe as being for log files what Snort is for network traffic and YARA is for files. Sigma rules can be converted into queries for many SIEM platforms, so a detection is not locked to one product.
QHow do we reduce false positives without missing attacks?
Make exceptions narrow and documented: exclude the specific process, path and account that cause the noise rather than a whole host or subnet, add context instead of raising thresholds, and keep every exception in version control with a reason, an owner and a review date.
QHow do we know a detection still works?
Re-run its validation, a safe reproduction of the attack it is meant to catch, on a schedule and after any change to the log source it depends on. A detection that has not fired in a long time may be fine or may be broken; only a test tells you which.
QWhy detect techniques instead of indicators?
Indicators such as file hashes and IP addresses are cheap for an attacker to change, so a detection based on them catches one campaign. Techniques reflect how the attacker operates and are expensive to change, so a behaviour-based detection catches every tool that uses the technique.
QHow should detection coverage be measured?
Against the ATT&CK techniques you have decided matter most, counting a technique as covered only when its detection has been validated successfully within a recent period. A heat map of rules that exist but were never tested overstates coverage.
QCan a managed SOC do detection engineering for us?
Yes, if the arrangement includes it explicitly. Ask who writes and owns detections specific to your environment, how they are tested, whether you can see the logic, and how findings from your penetration tests become new rules.
QHow long must logs be kept in India?
CERT-In's directions of 28 April 2022 require logs of all ICT systems to be kept securely for a rolling period of 180 days within Indian jurisdiction. Sector regulators and your own investigation needs may require longer.
Related reading
For the attack paths that make good detection candidates, see what an internal penetration test finds. For the wider operating model, the SOC incident management playbook and managed versus in-house SOC. For measuring the SOC as a whole, why a SOC maturity assessment matters.
About Adayptus
Adayptus Consulting Private Limited is a cybersecurity consultancy based in Noida, India. Our SOC work is built from the attacker's side: the detections we write come from techniques our own testers use, and each one is validated against a real reproduction of the attack before it goes live.
What we can do for you:
- Detection engineering and tuning. Use case development and tuning for the SIEM you already own: detections documented to the ADS standard, validated, and tuned with narrow, recorded exceptions.
- The platform underneath. SIEM implementation and optimisation, including the log source coverage every detection depends on.
- Running it. Managed or co-managed SOC operations, threat hunting for what rules miss, and purple team exercises that turn attack paths into validated detections.
Sources
- Palantir, Alerting and Detection Strategy framework.
- SigmaHQ, Sigma: generic signature format for SIEM systems.
- MITRE ATT&CK, T1558.003 Kerberoasting.
- CERT-In, Directions under section 70B(6) of the IT Act, 28 April 2022.
Adayptus Consulting
Security Operations, Adayptus
Adayptus Consulting Private Limited is a cybersecurity consultancy based in Noida, India, building SOC detections from the techniques its own testers use.
